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GENERAL GUIDELINES 


Background 


The First Episode Psychosis Services Fidelity Scale (FEPS-FS-1.0) is used to assess the degree to which mental 
health teams deliver specialized care comprised of a range of evidence-based practices to people experien- 
cing a first episode psychosis and their families. Program fidelity refers to the extent to which delivery of an 
intervention adheres to the protocol of an evidence-based program model. Fidelity scales provide a list of 
objective criteria by which a program is judged to adhere to a reference standard for the intervention. The 
scale can be conceptualized as an outcome measure for implementation research or as a quality measure for 
assessing structure and process indicators in health care (Donabedian, 1966). 


The First Episode Psychosis Services Fidelity Scale (FEPS-FS-1.0) 


The FEPS-FS was developed using a standardized methodology for developing fidelity scales (Bond et al., 
2000). The first stage involved three steps: (1) a systematic review of randomized controlled trials of first 
episode psychosis teams to identify the components of successful programs, (2) an assessment of the level of 
evidence supporting those components, and (3) a Delphi consensus process using international experts to 
identify the essential components (Addington, Mckenzie, Norman, Wang, & Bond, 2013). The second stage 
included three steps: (1) Developing descriptions of the core components, (2) defining behaviorally anchored 
criteria for ratings on a 5-point continuum for each component, and (3) development of a rating manual that 
described detailed procedures for data collection and scoring. The scale was tested for face validity, interrater 
reliability, and feasibility on programs in Canada and the United States . Results of the testing showed good 
interrater reliability and face validity. Compared with three other scales published (the Early Assessment 
and Support Alliance (EASA) scale, the RAISE Connection fidelity tool, and the EDEN scale, developed for 
Evaluating the Development and Impact of Early Intervention Services in England), the FEPS-FS has fewer 
components but the highest proportion of components common to all these scales (D. E. Addington et al., 
2016). The design of the scale and the evidence base upon which it was developed permits assessment of 
programs that are based on different models and operating in different health systems. 


The FEPS-FS-1.0 is a modified version of the original scale. Modifications were made in the course of and 
following two multi-center fidelity studies. The first, conducted in Canada, was a study of 9 programs in 
Ontario using an onsite review by trained clinicians, actively involved in health care delivery and health 
care evaluators (Durbin et al., 2019). The second study was a representative longitudinal cohort study of 36 
US programs using a central remote assessment team and phone interview. The reliability of 33 items of the 
revised scale were shown to be good to excellent (Addington, Noel, Landers, & Bond, 2020). After the stud- 
ies were completed, minor editorial changes were made to improve the wording of components and rating 


criteria. Two new components reflecting components missed in the original scale were added to form the 35 
components of the FEPS-FS 1.0. The two additional items included item 10, The age range served by the pro- 
gram, and item 35, Attention to fidelity. These are clear and concrete items that do not affect the reliability 
of the FEPS-FS 1.0 compared with the FEPS-FS revised version. 


The scale is available in two forms: a 35-item Team scale and a 20-item individual patient scale. 
1. The team scale can be rated in one of three ways: 
1.1. Site visit when expert reviewers visit a site. 


1.2. Remote assessment. Data is collected at the site from health record review and 
administrative data and interviews are conducted from a central site. 


1.3. Self Report. The site collects the data, interviews staff, and reports the results. 


2. The individual scale is focused on the services offered to an individual patient. The scale 
assesses the service components received by the patient (Appendix G). This version of the 
scale was developed to assess and quantify the quality of the care received by individual 
patients who did not receive care from a team-based service. It was originally designed for 
use in a randomized controlled study comparing services offered in standard care versus 
specialized team care. This version of the scale has not been tested in research studies. 


The scale was used as a self-report measure in a national study in Italy (Addington et al., 2020). Instructions 
on how to use the FEPS-FS 1.0 as a self-report measure are included in Appendix H. The self-report version 
does not differ from the observer-assessed version of the scale. The reliability of using the measure as a 
self-report measure has not been tested. Training in use of the scale is recommended for using the scale as a 
self-report measure just as it is for use by an external assessor. 


The terminology used in mental health services varies across countries, service delivery organizations, and 
the professional groups delivering the services. The purpose of the scale is to assess the quality of services 
received, not to focus on terminology. For example, the scale and the manual refer consistently to patients, 
but not clients. Fidelity raters should use the terminology of the programs and services that they are evaluat- 
ing. Thus, if the program or service uses the term clients rather than patients, the rater should use the term 
clients. Professionals delivering the services are also described using different terms. The term case manager 
means different things in different programs. In one program, the case manager does outreach and connects 
the patient to community services. In another program the case manager is the person primarily involved in 
coordinating the care within the team and perhaps for delivering a specific component of the services. It is 
a task of the fidelity rater to identify the services delivered by the different professional groups within their 
program and to use the program’s own terminology to identify the various staff. 


Different programs serve different patient diagnostic groups. This is a challenge for the fidelity rater, the 
fidelity sponsor, and the program. The research on team-based care for people with a first episode psychosis 
has focused on those with a first episode of a schizophrenia spectrum disorder. However, some programs 
also serve those with an attenuated psychosis syndrome (also known as the clinical high risk or ultra high 
risk groups), bipolar disorder, and/or a major depressive disorder with psychotic features. These different 
groups require different treatments, and the fidelity scale can be used in a way that adjusts for these different 
patient groups. These adjustments are defined as general principles and specific components for specific 
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disorders. It is up to the funders of specific fidelity reviews to determine the goals and objectives of the 
review. The most common focus for the fidelity reviews is for patients with a first episode of a schizophrenia 
spectrum disorder not including the attenuated risk syndrome. If that is the situation, the health records 
should be selected to reflect this population. 


General Principles: 


Reliable initial diagnosis. 

Record numbers in groups. 

Review health records for population of interest. 
Exclude patients not in population of interest. 


Examine each group with sufficient numbers independently. 


If there is a reason to assess fidelity for other groups, it is important to establish that there are sufficient 
numbers in those groups. To establish fidelity for these groups some components need to be dropped, as 
shown in the following table. 


PATIENT GROUP ITEMS TO OMIT 


Recent Onset (0-5) years None 
Schizophrenia Spectrum disorders 


Bipolar 19 Initial antipsychotic 
20 Antipsychotic dosing 
21 Clozapine 

Attenuated Psychosis Syndrome 13 Early Intervention 


19 Initial antipsychotic 
20 Antipsychotic dosing 
21 Clozapine 


The FEPS-FS is not designed to be a substitute for more detailed operating guidelines including coun- 
try-specific or health-system-specific guidelines, such as provincial or state guidelines. The scale has been 
designed to focus on evidence-based services that are specific to or adapted to individuals with a first episode 
psychosis. Therefore, important general health system policies such as those addressing cultural adaptation, 
legal requirements for privacy, notification of dangerousness, or requirements for involuntary treatment are 
not addressed. 
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FEPS-FS-1.0 Manual 


The First Episode Psychosis Services Fidelity Scale Manual provides a guide for scoring the FEPS-FS-1.0 and 
is designed to increase the reliability (consistency) of ratings across different sites and assessors. 


The Manual provides the following: 
e A definition and rationale for each component in the Fidelity Scale. 
e A list of data sources to inform the ratings for each component. 
e Decision rules to help score each component correctly. 
e Site interview data-collecting tools: interview guides, health record abstraction guide. 


e Site fidelity assessment preparation guide. 


Overview of the Scale 


The FEPS-FS 1.0 (see Appendix F) contains 35 program-specific components. Each component on the scale 
is rated on a 5-point scale ranging from 1 to 5. In the language of implementation this ranges from 1, “Not 
implemented” to 5, “Fully implemented”. In the language of quality this ranges from 1, “Poor Quality” to 5, 
“Excellent Quality”. In both languages, a 4 means good or satisfactory. The standards used to establish the 
anchors for the “fully implemented” ratings were determined through a variety of expert sources. The scale 
assesses the services received by program patients, the training received by the care providers, and how the 
team works together to engage and retain patients and deliver coordinated and evidence-based care. 


Services Delivered and Staff Roles 


Staffing patterns, professional designations, and individual roles vary significantly from one organization to 
another, and across health care jurisdictions and countries. To address this, the FEPS-FS-1.0 ratings focus on 
assessing the services received by the patients, rather than professional designation of the person who deliv- 
ers the service. In practice, assessors need to adapt questions to fit with the staffing pattern of the program 
being reviewed. For example, Cognitive Behavioral Therapy (CBT) might be delivered by a psychologist or 
by a trained counsellor who is a social worker by profession. Case management may be provided by a mental 
health professional who is called a case manager or care coordinator, or by someone called a counsellor or 
recovery coach who may have additional roles such as CBT or individual resiliency training. The scale also 
assesses the professional training of the staff and the specific training received in order to fulfil their role on 
the team and deliver the services provided to patients. 


Training Fidelity Assessors 


Trained assessors should conduct the fidelity assessments. Training varies depending on the experience and 
knowledge of the fidelity rater. It usually involves a two-day training program followed by two teleconferen- 
ces with the trainer to discuss consensus ratings of real programs. The training can be delivered in-person 
or remotely. 
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The first day of training addresses the following: 


The scale development process 

Review of individual components 

Review of the Manual 

Review of the process used for a fidelity assessment 


Review of best practices for implementing fidelity measures and other health care quality 
indicators 


The second day’s training involves case-based training. 


After conducting their first few fidelity assessments, raters should have the opportunity to review the process 


and the ratings with the trainer. 


Fidelity Assessment Process 


The scale can be used for onsite fidelity reviews, remote fidelity assessment, and self-assessment. The scale 
was originally designed and first evaluated using an expert in-person site interview method. It has since 
been adapted and tested for remote assessments including both remote data collection and staff interviews. 
The fidelity scale has also been used as a self-report measure. The scale and the recommended sources of data 


are the same in each data collection method. 


Preparing for a Site Fidelity Assessment 


Fidelity assessments require advanced preparation by all participants to ensure that assessors have time to 


speak with different program stakeholders and receive the information necessary to make the ratings. 


Assessor Role: 


Review the fidelity Manual, scale, and data collection tools in advance of the site 
assessment. 


Review any documents sent by the site in advance (see data source #1 in Data Collection 
and Data Sources section below for more detail). 


Complete any required training on ethical evaluation practices and health information 
privacy regulations. 


Communicate with the central coordinating unit to schedule the assessment date and 
organize the phone-in schedule. 


Ensure all necessary paperwork related to privacy/confidentiality is completed. 
(Requirements will vary from program to program.) 


Note: The assessors should meet in advance of the site assessment by teleconference to 
introduce themselves, discuss initial impressions from advanced materials, and confirm 
roles/approach for interviews. This can be conducted by phone. 


Arrange a specific time to train the team leader and the health record abstractor. 


General Guidelines 


e Interviews for a single program should be scheduled for one day and should start with the 
interview with the program manager. 


Program Role: 
e Identify the lead contact with the fidelity review team. 


e Create a schedule for site interview, line up staff interviews, pull active patient health 
records, and send assessors any important documents/reports. (For full instructions refer 
to Appendix A: Preparing for a Fidelity Assessment.) 


e Ensure all necessary internal approvals (i.e., administrative, ethics) are in place and 
required paperwork has been completed by the assessors. 


e Identify the person responsible for health record abstraction. 


e Communicate with the central team to schedule the assessment date. 


Central Team Role: 


e Liaise with the site and assessors to organize and schedule the site visits. 


e Support ethics and privacy processes. 


The most successful fidelity assessments are those in which there is a shared goal among the assessors and 
the program site to understand how the program is progressing and delivering evidence-based practices. 


Data Collection and Data Sources 


A detailed guide for programs undergoing a fidelity review can be found in: Appendix A: Preparing for 
a Fidelity Assessment. 

The assessor team will need to review three data sources: existing documents and administrative data sent in 
advance, data abstracted from the health records of active patients, and interviews with staff and patients. A 
schedule for the interviews or site visit will be prepared in advance by the site to ensure the evaluation runs 
smoothly 


1. Existing documents and administrative data 

Documents including policies, practices, detailed program description (including all program components), 
education materials, and routine reports/admin data (e.g., staff FTEs, admission and discharge statistics, 
etc.) should be provided by the program in advance. The full list of documents to include can be found in 
Appendix A: Preparing for a Fidelity Assessment. These documents need to be reviewed before the site 
assessment. 


NOTE: Only aggregate, de-identified data should be shared in advance. 


2. Data abstraction from active health records 
a. Ten active, randomly selected health records of patients who have spent one year in the 
program. The health record data abstraction will require approximately 4 hours. The central 
team will work with each site to develop a randomization process. The records should be 
selected from patients who have been receiving services for at least one year to ensure patients 
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have had adequate time to receive the services. Where programs are new or so small that 
they do not have 10 patients who have received services for a year, the assessment of the 
health records needs to be reconsidered in light of the purpose of the fidelity review. If the 
purpose of the study is to strictly compare programs against a standard which requires a size 
of service that has a meaningful impact on community services, the small program can be 
rated according to the proportion of 10 records that meet fidelity criteria. If the purpose of 
the fidelity review is to check on the processes of care as a new program is being established, 
the rating can be calculated based on the proportion of health records of patients who have 
received care for one year. The health record review can be undertaken by a local abstractor 
who will require brief training from the team responsible for the fidelity review and an 
orientation to the health record by a clinical team member of the first episode psychosis 
service. More detail on selecting patient health records can be found in Appendix A: 
Preparing for a Fidelity Assessment. 


b. Health records of the last five patients who have had a hospital admission after joining the 
program. The focus of this health record abstraction is to identify the data required to rate 
Component 31, Communication Between FEPS and Inpatient Service, and Component 32, 
Timely Contact after Hospital Discharge. Where programs are new or so small that they do 
not have five patients who have been hospitalized, the assessment of the health records needs 
to be reconsidered in light of the purpose of the fidelity review. 


Any program requirements to support privacy or access from both an ethical and logistical perspective 
should also be confirmed ahead of time. The Health Record Review Checklist (Appendix B) can be used 
to extract the relevant data from the health records. It is designed to be completed without the use of any 
information that could be used for personal identification. 


3. Interviews with staff 

A range of program staff should be interviewed during the fidelity assessment. At all programs it is import- 
ant to interview the program manager/team leader, one case manager/care coordinator, the psychiatrist or 
other prescriber, and the supported employment specialist. These interviews will be organized in advance by 
the program, and interviews are conducted individually. The specific configuration of interviews with staff 
will depend on the team composition and functions. The assessor should interview all staff needed to obtain 
the necessary data to complete fidelity ratings. 


See Appendix C for the Fidelity Interview Guide. It is helpful to review the interview schedule with the 
site lead at the beginning of the first day of the site visit to clarify the roles of each staff member who will be 
interviewed. 


It is the responsibility of the assessors to ensure that, when required, informed consent is received at the 
beginning of each interview and to make clear that participation is voluntary and that a decision to not 
participate will not affect staff member’s employment in the program. Site-specific consent forms will be 
provided to each assessor team, and the signed consent forms will remain onsite and stored by the program. 
Detailed notes should be kept during the interviews to support the final fidelity ratings. 


NOTE: No identifying information (e.g., names of staff, patients, or families) should be included in assessor 
notes. Notes also should not include any comments on individual work performance. 
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Confidentiality and Data Storage 


It is important to ensure that proper confidentiality protocols are in place for each site interview and that any 
data collected is stored in an appropriate manner. Prior to or at the beginning of the site review, all assessors 
must sign the necessary confidentiality forms. These vary depending on the data collected during the site 
review and may include notes from the patient, family and staff interviews, notes from the staff meeting, and 
the completed Health Record Review Checklist (Appendix B), as well as any documents or reports shared by 
the site. The information collected should not include any names or references to individual patients, family 
members, or staff or personal health information. No health records or identifiable information should leave 
the site. 


The information will be kept in a secure location by the assessor (if paper-based, in a locked cabinet/office, 
and if electronic, in a password-protected file) until the final fidelity report has been produced. At that point, 
the assessors will send the files to an appropriate site storage depending on the purpose and regulations 
governing that purpose. If the assessment is part of a research project, the storage follows research require- 
ments. If it is a quality improvement project, then local storage requirements should be in alignment with 
local procedures. Before sending the documents, assessors should complete a final review of the documents 
to ensure no individuals are identified. 


How to Rate Components and Triangulate Across Data Sources 


It is the task of the assessors to review and synthesize all data collected to determine the score for each com- 
ponent on the scale and to complete the final report. 


How to rate components: 


e Ratings should be made based on the scale as it is written. Any concern that the rating 
does not accurately reflect program practice should be captured in the comments section 
of the final report. 


e ‘The scale ratings are based on current behavior and activities, not planned or intended 
behavior. 


e For multi-site programs, if service delivery differs across program sites, consider rating 
the sites separately or rate according to the higher performing program and describe the 
discrepancy in the comments section. 


e Ifa period of time is not specified, then the rating can be based on service delivered at any 
point during the period of care. 


Which data source to use: 

The next section of this Manual lists all relevant data sources for each component. All the listed data sources 
for each component can be used to complete the comments section of the report, but we have included 
instructions on which data source is the suggested primary source to identify the rating for that component. 


e Wherever possible, program administrative data or health records should be used to 
determine the final component rating. 
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e In general, if care related to a component is typically documented in the health record and 
is provided to all patients, then the health record is used as the data source. In this case, 
if the component is not documented in an individual health record, we assume it did not 
happen. It is of course possible that it did occur and simply was not documented. This 
possibility can be discussed in the comments but should not impact the rating. 


e However, if a component is not routinely documented in the health record, or is provided 
only to a minority of patients, an alternative data source (e.g., staff interviews) can be used 
to support the rating. If this is a possibility, interviews will be listed as the appropriate 
data source for that component. For example, if cognitive behavioral therapy is not 
documented in the health record, the record cannot be used as a data source. 


e All data sources may provide important contextual information that should be included in 
the comments section of the final report (though it may not necessarily impact the rating). 
For example, if the rating for a component is low and the policy review indicates that no 
procedures are in place to support the component, this may be a practice improvement 
area to flag. On the other hand, if the rating is low and the processes seem appropriate, it 
may be a documentation issue, and this may be flagged. 


After Your Fidelity Review 


If possible, time should be set aside after the interviews and after the review of the documentation to review 
data to explore discrepancies. Queries can be addressed with a follow-up email or call to the site. It is critical 
that the interview rating and documentation be reviewed while the information is still fresh. 


After the site interview a consensus rating meeting may be scheduled with the assessors and a fidelity expert 
if available. The assessors present their ratings and rationale, obtain feedback from the expert, discuss dif- 
ferences, and agree on a final consensus rating. The fidelity expert will ensure scoring decisions are in-line 
with the intended use of the scale. 


The assessors can prepare the fidelity ratings and/or report and send to the program or research team with- 
in an agreed upon period. Two weeks after the consensus rating meeting is a reasonable time frame. A 
Feedback Report Template can be found in Appendix H. The report can include: 


e A high-level overview of findings, highlighting program strengths and opportunities for 
improvement 


e Component fidelity scores 
e Data sources used to determine each score 
e Any contradictions between data sources 


e Any additional relevant contextual information that might explain the score (e.g., 
problems with outreach in remote regions) 


e Specific additional information requested per component (clearly specified in template) 


e Additional assessor comments (e.g., Was this component difficult to rate? Do you feel the 
rating is valuable/ reflects program practice?) 
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COMPONENT CRITERIA AND RATINGS 


1. Practicing Team Leader 


Definition: Program staff receive both administrative leadership and clinical supervision. These roles may 
be held by the same individual (likely a manager or team leader) or by two different individuals. To get the 
highest rating, the individual who provides clinical supervision should also provide some direct clinical 
services in the first episode psychosis program. It is not required that these individuals have a master’s-level 
education, but in some services, this is a requirement for this role. 


Rationale: Longitudinal studies of implementing best practices show a significant correlation between the 
presence of a practicing team leader and program fidelity. 


Component Scoring: 


e Data source to use for rating: Interviews 


e Component response rating: If the team leader (or other individual) provides 
administrative direction and clinical supervision to all staff as well as providing some 
direct clinical service, code the component as ‘5’. 


e Direct clinical service: The supervisor is providing a service and not just observing another 
clinician as part of supervision. It does not have to be functioning as a care coordinator or 
case manager with a caseload. It can be completing the intakes or family psychoeducation 
or offering individual therapy. 


Comments: 
e Additional data sources to support comments: 
- Interviews with the team leader and clinicians. 
e Specific information to include: 


- Does the team leader hold multiple positions (i.e., Team Leader role for other 
programs or formal role as intake person for the first episode psychosis program)? 
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2. Patient-to-Provider Ratio 


Definition: There is a target ratio of patients to Full Time Equivalent (FTE) clinical staff. 
Rationale: Optimal ratios have been reported in the range of 15-20 cases per FTE. 


Component Scoring: 


e Data source to use for rating: Interviews 


e Component response coding: If interviews indicate that the caseload ratio is 20:1 or less, 
rate the component as ‘5’. 


To calculate: Divide the total number of currently registered patients by the total number of clinical FTEs. 
A Full Time Equivalent (FTE) is a unit that indicates the workload of an employed person in a way that 
makes workloads comparable across various contexts. FTE is often used to measure a worker’s involvement 
in a program. The calculation of full-time equivalent (FTE) is an employee’s scheduled hours divided by the 
employer’s hours for a full-time workweek. When an employer has a 40-hour workweek, employees who 
are scheduled to work 40 hours per week are 1.0 FTEs. Employees scheduled to work 20 hours per week are 
0.5 FTEs, and 0.1 FTE represents one half-day of work. In the FEPS-FS, FTEs include all direct clinical care 
staff. Direct care staff include family support worker, peer support worker, nurse, employment specialist, 
addiction specialist, psychologist, therapist, and case manager. Do not include residents/interns even if they 
have their own caseload. Do not include any staff members who have no clinical role. These may include 
administrators, data managers, managers, and researchers. Do not include prescribers. If the team leader 
has both an administrative role and a caseload, in the calculation only include the percentage of time the 
team leader dedicates to clinical services. 


Comments: 
e Additional data sources to support comments: 
- Interviews with team leader, clinicians, and Supported Employment specialist 
- Document review 


- Policy and practice documents 


3. Services Delivered by Team 


Definition: Includes qualified professionals to provide both case management and specific service compon- 
ents including: 


i. Case management/care coordination 
2. Health services (registering with primary care, weight and metabolic monitoring) 


3. Psychotherapies including Cognitive Behavioral Therapy (CBT) and Motivational 
Enhancement (ME) (aka Motivational Interviewing) 


4. Substance use management 


5. Supported employment/Supported education 
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6. Family education and support 
7. Patient psychoeducation 


8. Pharmacotherapy 


The focus for rating this component is the services received by the patients rather than the professions of the 
providers. Some of these components may be delivered by a staff member paid by another program but can 
still be included if that staff member is an active participant in the patient’s multidisciplinary team. More 
than one function may be provided by the same individual. Rating the service as present is separate from 
the rating of the quality of the service, which is covered by the assessment of each component. A minimum 
rating of ‘2’ on the specific component score should be required to define the service as present. 


Rationale: Patients and families benefit from a range of services which need to be coordinated to deliver 
consistent care. 


Component Scoring: 


e Data source to use for rating: Interviews and document review. 
e Most services are defined under the rating for that component. 


e Component response coding: If team members provide all the listed services, code the 
component as ‘5’. 


Comments: 
e Additional data sources to support comments: 
- Interviews with team leader and clinicians. 


- Document review (program staff role descriptions) 


4. Assigned Case Manager/Care Coordinator 


Definition: Patient has an assigned case manager/care coordinator (individual may have a different title at 
different programs) who is a professional qualified clinician in nursing, psychology, social work, or occupa- 
tional therapy. 


Case managers (or “care managers/coordinators”) are staff members on coordinated specialty care teams 
who assess and address the individual and unique needs of each patient, providing direct services or making 
referrals with follow-up ensuring that individualized needs are addressed. Case managers are part of the 
team, attend team meetings and coordinate care for patients. Case management responsibilities may include 
mental health counseling, skills training, financial counseling, housing assistance, substance abuse counsel- 
ing and treatment, and family counseling. 


Rationale: Case management is a component of all treatment guidelines. 
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Component Scoring: 


e Data source to use for rating: Health record review or administrative data 


e Component response coding: If 80% or more patients have been assigned a case manager, 
code the component as ‘5’. 


Comments: 
e Additional data sources to support comments: 


- Interviews with team leader and clinicians 


5. Psychiatrist Caseload 


Definition: Access to psychiatry is an important component of the first episode psychosis treatment model, 
and each patient should be seen up to once a week by a psychiatrist if necessary. It is important that psych- 
iatrists maintain low caseloads to facilitate this frequency. Other appropriately trained prescribers can take 
responsibility for pharmacotherapy. Appropriately trained means that the licensed prescriber can evaluate 
symptoms and functioning, is aware of the available antipsychotics, their risks and benefits, as well as the 
indications for other psychopharmacological interventions in psychosis. Moreover, they should be able to 
monitor and manage the side-effects. 


Rationale: Implementation guidelines specify the need for psychiatrists or other appropriately trained pro- 
viders to be part of the team. 


Component Scoring: 


e Data source to use for rating: Interviews with team leader and prescriber, and program 
documents 


e Component response coding: If interviews indicate that psychiatrist works with <29 
patients per 0.2 FTE, code the component as ‘5’, 


To calculate: 
1. Add up the total psychiatry FTEs available to the program per week. 
2. Divide the total number of enrolled patients by the total psychiatry FTEs. 
3. Multiply that number by 0.2. 


For example, if you have 30 currently registered patients and a 0.1 psychiatrist FTE, your caseload is 60 per 
0.2 psychiatry FTE. 


Use the prescriber with the highest ratio who serves at least 30% of the full caseload to code the component. 


If the caseload is divided more or less equally across multiple prescribers who have equivalent time alloca- 
tions but variable caseloads, calculate the patient-to-prescriber FTE ratio for each prescriber. 


14 First Episode Psychosis Services Fidelity Scale and Manual 


For example, a program has 43 active patients and 3 prescribers. Prescriber 1 is 0.1 FTE and has a caseload 
of 16, Prescriber 2 is 0.2 FTE and has a caseload of 25, and Prescriber 3 is 0.1 FTE and has a caseload of 20. 
Converting their caseload ratios to a 0.2 FTE: 

Prescriber 1 — 0.2: 16 

Prescriber 2 — 0.2: 25 

Prescriber 3 — 0.1: 20 

Combined (FTE 0.5): 65 

Prescriber caseload 25/ 0.2 FTE score 5 


Comments: 
e Additional data sources to support comments: 
- Staff FTE documents 
e Specific information to include: 
- How many FTEs of psychiatry do you have available? 


- Ifa patient needs to be seen every two weeks by a psychiatrist, is that possible? 


6. Psychiatrist Role on Team 


Definition: Psychiatrists/licensed prescribers are team members who provide direct clinical services to indi- 
vidual patients and their families. They share an integrated health record with other team members. In addi- 
tion, as team members they attend team meetings, may see patients with other clinicians and are accessible 
for consultation by team during the work week. 


Rationale: Program guidelines specify the need to have a psychiatrist/prescriber as part of the team. 


Component Scoring: 
e Data source to use for rating: Interviews with the team leader and psychiatrist/prescriber 


e Component response coding: If interviews indicate that the psychiatrist/prescriber attends 
team meetings, sees patients with other clinicians, shares the team health record, and is 
available for consultations with staff during the work week, code the component as ‘5. 


e ‘The psychiatrist/prescriber participation must be consistent. Appropriate and accountable 
funding for these activities reinforces this pattern of practice and strengthens the evidence 
for the rating. 


e ‘The psychiatrist must attend three or more team meetings per month to meet the criterion 
of attendance at team meetings. 


e Some programs may use outside prescribers who are not involved as team members or 
who do not share the same health record for certain patients. If this affects <20% of the 
current caseload, this should be noted as deficiency in any written review. If it affects >20% 
of the current caseload the team should not receive the point for sharing the health record 
with other team members. 
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NOTE: If there are multiple psychiatrists/prescribers who work with the program and who have different prac- 
tices regarding their participation on the team, the rating should be made based on the main psychiatrist/ 
prescriber (the one with the most patients) and discrepancies can be discussed in the comments. 


7. Weekly Multi-Disciplinary Team Meetings 


Definition: Multi-disciplinary team meetings are conducted weekly to discuss the following: 
1. Case review (new admissions and caseloads) 
2. Assessment and treatment planning 
3. Complex cases 
4 


Termination of services 


Rationale: Regular team meetings are conducted to review the status of first episode psychosis patients and 
foster staff communication. 


Component Scoring: 


e Data source to use for rating: Team Leader Interview 
e All team members must be present. 


e Ifthe psychiatrist is absent this still counts as a team meeting, but the absence is rated 
under the psychiatrist role. To be present the psychiatrist must attend three or more of the 
team meetings in a month. 


e Component response coding: If the team meetings are conducted weekly, and involve 
discussion of all four issues, code the component as ‘5’. 


e Admissions and caseload refer to discussion of new cases and to maintaining an adequate 
number of cases or reviewing premature discharges. 


e Termination of services refers to the clinical detail involved in transition in care after the 
end of the program. 


Comments: 
e Additional data sources to support comments: 


- Interview with case manager 


8. Explicit Diagnostic Admission Criteria 


Definition: The FEP program has a clearly identified mandate to serve specific diagnostic groups with a 
psychosis and uses measurable and operationally defined criteria to select appropriate referrals. There exists 
a consistent process for screening and documenting uncertain cases and those with co-morbid substance 
use. The exact diagnostic criteria admitted vary according to the program. Typically, all include those with 
a schizophrenia spectrum disorder, and some include those with a bipolar disorder or major depressive 
disorder with psychotic features. Some also serve those with a DSM-5 Attenuated Psychosis Syndrome, 
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also known as those at Clinical High Risk. It is necessary to know the proportions of these groups in the 
program, since this affects the components that assess pharmacotherapy. 


Rationale: The program needs explicit criteria to conduct effective evaluations and make comparisons with 
similar programs. 


Component Scoring: 


e Data sources to use for rating: 
- Interview with team leader and document review 
- Policy document 
e Specific information to include: 
- Describe the program inclusion and exclusion criteria for admission. 
- Provide the number of people enrolled in the program by diagnostic group. 


¢ Component response coding: If over 90% of the patients served meet admission criteria, 
code the component as ‘5’, 


e Patients with ambiguous diagnoses: Count patients with ambiguous diagnoses who later 
meet criteria for a non-included diagnosis as meeting criteria for program entry. 


9. Population Served 


Definition: This component evaluates the extent to which first episode psychosis programs are meeting 
population need. In other words, are they serving the number of new patients expected, based on the inci- 
dence of new cases expected from the population in the catchment area? Catchment area may be defined by 
city, county, or state or region. For guidance, we have included recommended incidence rates based upon 
data from a systematic review of the literature. If there is data available that describes the precise incidence 
of new cases in your service area, this should be used. 


Rationale: Programs should enroll patients with a schizophrenia spectrum disorder at the annual incidence 
rate of 16:100,000 people (McGrath, Saha, Chant, & Welham, 2008). Use local data on incidence if available; 
for example, in England a data base provides specific incidence data for each region (Kirkbride et al., 2013). 


Component response coding: If the program serves 280% of incident cases expected within the region served 
by the program, score ‘5’. 


If the program or the service provider organization does not compare the admission rate to the expected 
incidence rate, rate the component as ‘1’. 


Comments: 
e Data sources to support comments: 
- Interviews with the team leader 
- Program administrative data on new admissions 


- Publicly available population statistics* 
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e Specific information to include: 


- Number of new patients admitted in the past 12 months per 100,000 population 


*Use a government-based census source (e.g., www.census.gov) to find the population of catchment area. 
NOTE: Calculation should be based on the typical number of unique patients admitted in the past 12-month 
period and the population of the program’s catchment area. Information on new patients and estimated bound- 
aries of the program catchment area can be obtained from the program. 


To calculate: 


1. Gather necessary numbers: 
a. Identify the number of new patients admitted to the program in the past 12 months. 
b. Identify program catchment area boundaries. 


2. To calculate the proportion of total potential patients served: 
a. Look up catchment area population size. 
b. Divide the number of new patients by the total catchment area population. 
c. Multiply this number by 100,000. 
d. The result is the number new cases per 100,000 population. 


10. Age Range Served 
Definition: The age range accepted by the program. 
Rationale: The epidemiology of schizophrenia indicates that the age range of onset covers the age range from 


early teens to 65. Programs that restrict the age range exclude older patients who are more often female and 
more likely to be married. 


e Data Source: Program policies, team leader interview 


11. Duration of First Episode Psychosis (FEP) Program 


Definition: The mandate of the program is to serve patients for a specified period. 
Rationale: Due to the nature of the early course of illness, patients and families often benefit from receiving 
first episode psychosis services for at least three years. The level of services required varies between individ- 


ual patients and their families and over time. 


Component Scoring: 


e Data source to use for rating: Interviews and program policies review 


e Component response coding: If the intended duration of the service is more than three 
years, code the component as ‘5’. For this component, duration is based on the program 
policy and stated mandate (though length of time in program for individual patients may vary). 
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Comments: 


e Additional data sources to support comments: 


- Interview with team leader and clinicians 


- Program policy documentation review 


12. Targeted Education to Health/Social Service/Community Groups 


Definition: Provision of information to first-contact professionals, including family physicians, school and 


post-secondary counseling services, youth social service agencies, community mental health services, police 


services, hospital emergency rooms, and other community organizations. Public education may be provid- 
ed by any source within the program or network. 


Rationale: Early identification approaches that involve proactively seeking out patients with early psychosis 
reduce the duration of untreated psychosis. Enhancing the education, communication, and liaison with 


health and service providers who may identify or treat patients with early psychosis will reduce the duration 
of untreated psychosis. 


Component Scoring: 


Data source to use for rating: Interviews and document review 


Component response coding: If community education is provided to service providers more 
than 12 times in previous 12 months, code the component as ‘5’. 


Hospital liaison is rated under component 33. It does not get rated under this component 
unless it is for emergency room contacts. 


Repeated visits to one site: It is reasonable to visit a site more than once given staff turnover 
leads to loss of information. Ongoing contact with a site around clinical cases does not 
count. 


Within-agency education: Do not count in-house agency education sessions. 


Do not count inpatient unit education since this is part of component 33, Communication 
between FEP and inpatient services. Furthermore, it does not promote community 
awareness. 


NOTE: This should be based on face-to-face education. Although potentially useful, activities such as poster 
campaigns, mass mail outs, phone calls, faxing, or emailing should not be counted towards this component. 


Comments: 


Additional data sources to support comments: 
- Interviews with team leader, clinician, and outreach coordinator 
- Public education materials (e.g., slides, handouts, speaking schedule, etc.) 


- Program policy 
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13. Early Intervention 


Definition: The proportion of first episode psychosis patients who have been hospitalized prior to FEPS 
program admission. 


Rationale: The proportion of first episode psychosis patients who have been hospitalized prior to admission 
to the FEP services reflects success in early intervention. 


Component Scoring: If no more than 19% of the current caseload were hospitalized in an inpatient psychi- 
atric unit prior to enrollment, code the component as ‘5’. The numerator for this measure is the number of 
patients admitted in the last year. 


NOTE: Do not include patients categorized as “clinical high risk” in calculating the proportion. 


Comments: 
e Data sources to support comments: 
- Interviews with the team leader 


- Program administrative data 


14. Timely Contact with Referred Individual 


Definition: Patients should receive an in-person appointment within two weeks of referral (10 business days) 
to the program. The time starts on the date that the referral from the original referral source is received at 
the program and ends when the patient has an in-person appointment. To meet criteria, appointments must 
be face-to-face and must include the opportunity to start some element of treatment. Treatment can include 
any activities that begin with the engagement of the patient and the process of recovery. This can include the 
first intake appointment where initial assessment, education, and the engagement process begin. The rating 
is based on appointments attended, not appointments scheduled or offered. 


Rationale: Patients who experience a first episode of psychosis require urgent or emergent care. Treatment 
should be initiated within two weeks of referral. Clear intake and admissions procedures support receipt of 


a timely initial assessment and treatment. 


Component Scoring: 


e Data source to use for rating: Health record review or administrative data (If program can 
pull all initial appointment data, the full dataset rather than the health record review can 
be used to assign the rating.) 


¢ Component response coding: If 80% or more patients receive a face-to-face appointment 
within two weeks, code the component as ‘5’. 


e Hospitalized patients: Some patients are first seen by FEPS staff in hospital. This would 
count as their initial assessment and there would be zero time between referral and 
assessment. 
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NOTE: If a patient is hospitalized after referral but before their first appointment, thereby preventing them 
from attending a first appointment within 2 weeks, they should be excluded from this calculation. If the patient 
is referred again to the program, the time is then calculated from the time of hospital discharge. 


Comments: 
e Additional data sources to support comments: 
- Interviews with program manager, case manager, or intake coordinator 


- Documented program policy and procedure 


15. Family Involvement in Assessments 


Definition: Service engages family in initial patient assessment to improve the quality of the assessment, 
and to engage both in the treatment program. Occasionally the patient is first seen alone. In these cases, if 
the family is seen within a month of the initial individual meeting, this can be counted as part of the initial 
assessment. 


Rationale: The engagement of individuals and families or other carers as partners in the assessment process 
and care improves the overall reliability of the assessment and may foster therapeutic alliance (e.g., helping 
choose targets for intervention). 


Component Scoring: 


e Data source to use for rating: Health record review or administrative data 


e Component response coding: If 80% or more of families are seen during initial assessment 
or within a month of that meeting, code the component as ‘5’. 


Comments: 
e Additional data sources to support comments: 


- Interviews with program manager, case manager, or clinicians responsible for initial 
assessments 


- Program policies and procedures for intake/assessment 


16. Comprehensive Clinical Assessment 


Definition: Each patient should have a comprehensive clinical assessment as part of their initial assessment. 
It should include the following: 


1. Time course of symptoms, change in functioning, and substance abuse (assessing whether 
there was change in functioning correlated with substance use changes or other changes) 


Recent changes in behavior 
Risk assessment of harm to self/others 


Mental status exam 


2 oe oe 


Psychiatric history 
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Premorbid functioning 
Comorbid medical illness 


Comorbid substance use 


we oN a 


Family History 


Rationale: Comprehensive assessments provide essential information for diagnosis, treatment decisions, and 
care planning. 


Component Scoring: 


e Data source to use for rating: Health record review, clinician interview. 


¢ Component response coding: If 8 of the 9 items above are assessed at enrollment for 80% or 
more of patients, code the component as ‘5’. 


e The comprehensive assessment may be completed over time, for example, as a combination 
of assessments from the screening interview to the initial assessment by the psychiatrist 
and other clinicians. The initial assessment must be completed within one month of 
enrollment. 


Comments: 
e Additional data sources to support comments: 
- Interview with the team leader 


- Policy and procedure manuals, assessment templates (if available) 


17. Comprehensive Psychosocial Needs Assessment 


Definition: Each patient should have an assessment of their psychosocial needs. The assessment should in- 
clude the following: 


1. Housing 
Employment 
Education 

Social support 
Financial support 
Primary care access 
Family support 


Past trauma 


CP OPN Aw AR wD 


Legal 


Rationale: Comprehensive psychosocial needs assessment provides essential information to support pa- 
tient-centered care planning to support recovery in the community. 
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Component Scoring: 
e Data source to use for rating: Health record review or clinician interview. 


¢ Component response coding: If eight or more psychosocial needs of patients are included in 
80% of needs assessments, code the component as ‘5’. 


Comments: 
e Additional data sources to support comments: 
- Relevant policy or protocol 
- Interview with the team leader 


e This component assesses the thoroughness of the assessment process. It is not meant to 
suggest that all these items should be included in a treatment plan. The treatment plan will 
reflect the needs identified and the preferences of the patient. 


18. Clinical Treatment Plan/Care Plan after Initial Assessment 


Definition: It is important that treatment plans/care plans reflect patient and family preference as well as 
clinical and psychosocial needs. Evidence that patient and staff collaborate to develop a treatment plan 
include: 

a. Patient’s signature on the plan 


Rationale: The clinical presentation and impact of psychosis is variable, and it is important that treatment is 
individualized. 


Component Scoring: 


e Data source to use for rating: Health record review, Clinician Interview 


e Component response coding: If 80% or more patients have an individualized treatment 
plan that reflects patient preference, code the component as ‘5’. 


Comments: 
e Additional data sources to support comments: 
- Interviews with team leader 


- Program policy/procedures 


19. Antipsychotic Medication Prescription 


Definition: After assessment confirms a diagnosis of a psychosis and the need for pharmacotherapy, anti- 
psychotic medication is prescribed with consideration given to patient preference. 


Rationale: Antipsychotic medication is an evidence-based core intervention to treat symptoms of psychosis 
and reduce relapse rates for patients with a first episode of a schizophrenia spectrum disorder. 
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Component Scoring: 


e Data source to use for rating: Health record review or administrative data 


e Component response coding: If 80% or more patients with a schizophrenia spectrum 
disorder receive a prescription for antipsychotic medication or enter the program already 
on an antipsychotic, code the component as ‘5’. 


e Do not count patients with a clinical high risk for developing a schizophrenia spectrum 
disorder (DSM-5 Specified Schizophrenia Spectrum Disorder Attenuated Psychosis 
Syndrome) or those with bipolar disorder. These patients do not contribute to the 
denominator. 


e This component refers to a prescription in the first month of starting the program. The 
dose may not be initiated at the full treatment dose. 


Comments: 
e Additional data sources to support comments: 
- Interviews with team leader, clinician, and prescriber 


- Program policy 


20. Antipsychotic Dosing within Recommendations for Individuals with 
Psychosis 


Definition: Antipsychotic medication dosing is within government-approved guidelines for second-genera- 
tion antipsychotics and between 200 and 800 Chlorpromazine Equivalents for first-generation antipsychotic 
medications at six months (See Appendix J for Medication Dosing Guidelines). 


Rationale: First episode psychosis patients tend to respond to lower doses of antipsychotic medications and 
are more sensitive to side-effects than multi-episode patients. At the same time, a proportion do not respond 
to medication at all, which means that in practice the full range of doses should be used. 


Component Scoring: 
e Data source to use for rating: Health record review or administrative data 


e Component response coding: If by six months of starting the program, 80% or more of 
patients are on a dose within the target range, code the component as ‘5’. 


e Multiple antipsychotics: If the patient is prescribed more than one antipsychotic, take the 
current dosage of each and divide them each by their maximum dosage. Add the fractions. 
If the total is >1, then the patient is not receiving antipsychotic dosing within guidelines. 


- Example 1: Patient is prescribed 15 mg aripiprazole and 4mg risperidone. Divide 
aripiprazole dosage by the maximum aripiprazole dose (15mg/30mg = 0.5) and divide 
risperidone dosage by the maximum risperidone dosage (4mg/16.0mg = 0.25). Add 
the two fractions (0.5 + 0.25 = 0.75). Total value (0.75) is < 1. Therefore, the prescribed 
dosage is within dosage guidelines. 
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- Example 2: Patient is prescribed 5 mg aripiprazole and 20mg olanzapine. Divide 
aripiprazole dosage by the maximum aripiprazole dose (5mg/30mg = 0.17) and divide 
olanzapine dosage by the maximum olanzapine dosage (20mg/20mg = 1.0). Add the 
two fractions (0.17 + 1.0 = 1.17). Total value (1.17) is >1. Therefore, the prescribed 
dosage is not within dosage guidelines. 


Comments: 
e Additional data sources to support comments: 
- Interview with prescriber 


- Program policy 


21. Clozapine for Medication-Resistant Symptoms 


Definition: Use of clozapine if patient does not respond adequately after two trials of antipsychotics (medi- 
cation resistance occurs in approximately 20% of cases with schizophrenia spectrum disorders). 


Treatment equivalent to 10 mg haloperidol, and over 3-month period. 


Rationale: Clozapine is indicated for the treatment of inadequate or non-response to first-line antipsychotics. 


Component Scoring: 


e Data source to use for rating: Clozapine monitoring enrollment records, administrative 
data or interviews (depending on what is available) 


e Component response coding: If more than 4% of caseload of patients with schizophrenia 
spectrum disorder are on clozapine, code the component as ‘3’. 


To calculate: 

Divide the total number of program patients with a schizophrenia spectrum disorder who are on clozapine 
by the total number of currently enrolled patients diagnosed with a schizophrenia spectrum disorder. 
NOTE: For programs that serve multiple groups of patients (i.e., schizophrenia spectrum disorders, bipolar 
disorder, depression, and clinical high risk or attenuated psychosis syndrome) the numerator and denomin- 
ator are exclusively patients with a schizophrenia spectrum disorder. 


Comments: 
e Additional data sources to support comments: 
- Interviews with prescriber 


- Ifthere is a discrepancy between the prescriber and documents prepared and 
submitted by the team leader, it is likely that the prescriber is more aware of the 
numbers due to the practical impact of having patients on clozapine. If the numbers 
are discrepant, provide the prescriber with the number provided by the team leader 
for clarification. Use the prescriber’s final determination as the count for the number 
of patients on clozapine. 


- Program policy 
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22. Patient Psychoeducation 


Definition: Provision of at least 12 episodes of patient psychoeducation in the first year. Psychoeducation re- 
fers to the provision of support, information, and management strategies related to familial, social, biologic- 
al, and pharmacological perspectives on illness. (See below for a detailed list of topics that may be addressed 
through psychoeducation.) An episode or session is usually 45-60 minutes covering a specific topic and its 
relevance to a patient. 


The person delivering the psychoeducation must have received formal training in patient psychoeducation 
as well as supervision. The training may have been part of their formal professional training or as continuing 
professional development (CPD). For both professional training and CPD, the training should comprise 
both learning and supervision. Attending courses without supervision is not evidence of effective training. 


Psychoeducation can be delivered individually or in a group and may be delivered by any member of the 
care team with formal training including peer support workers. A structured psychoeducation manual may 
be used, or psychoeducation may be delivered more informally, embedded as a component of case manage- 
ment. The RAISE research Individual Resiliency Manual sessions 1-3 and 7 are considered to cover psycho- 
education, as are modules 12 and 13 on health. 


Rationale: Psychoeducation is a way of providing information to patients to both engage them and support 
autonomy and recovery. 


Component Scoring: 


e Data source to use for rating: Health record review or administrative data (e.g., group 
attendance) 


- May be captured in structured documentation or in topics listed in progress notes, 
including: 


» developing coping and self-help strategies 

a developing resiliency 

» dealing with the symptoms of psychosis 

a activities of daily living 

a — educational/academic supports 

a — vocational/employment supports 

a housing supports 

a substance abuse supports 

a support in establishing social relationships or connections 
n peer support 

= income support, when necessary (i.e., benefits planning) 


a recreational supports 
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e Component response coding: If 80% or more patients participate in the equivalent of 12 
episodes of psychoeducation, code the component as ‘5. 


e Ideally the sessions are provided during the first year. That is one of the reasons for selecting 
health records of patients who have been in the program for at least one year. If the total 
of twelve sessions is only achieved in the second year this should be counted towards the 
rating. 


Comments: 
e Additional data sources to support comments: 
- Interviews with team leader and clinician 
- Psychoeducation manual or handouts 
e Specific information to include: 


- Are patients offered group or individual psychoeducation? 


- Is there a formal manual or curriculum that is followed? 


23. Family Education and Support 


Definition: Families should receive at least eight episodes of psychoeducation in the first year. Family psycho- 
education typically offers information on a range of topics including: 


e Information about psychosis 

e Information about medications 

e Information about coping with stress 
e Recognition and prevention of relapse 
e Collaboration with clinicians 

e Effective communication 


e Supporting recovery 


An episode or session is usually a 45-minute to one-hour session covering a specific topic and its relevance 
to a family. It can be delivered individually or in a group and may be delivered by the case manager or by 
a dedicated family worker. Evidence-based one-day training workshops can be part of a family education 
program. These typically last for four to six hours and can count for between 4 and 6 sessions, depending 
upon the schedule of the day. 


The person delivering the therapy must have received formal training in family psychoeducation and super- 
vision. The training may have been part of their formal professional training or as continuing professional 
development (CPD). For both professional training and CPD, the training should comprise both learning 
and supervision. Attending courses without supervision is not evidence of effective training. The Multifamily 
Group Treatment (MFGT) model is an evidence-based approach that offers a specific training course. The use 
ofa research-supported manualized program such as the RAISE Navigate Family Education Program Manual 
or the On Track New York Family Resource and Treatment Manual increases the confidence of a rating. 
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Rationale: Family psychoeducation is a robust contributor to lower relapse rates. 


Component Scoring: 


e Data source to use for rating: Health record review, interviews or administrative data (e.g., 
group attendance) 


e Component response coding: If 80% or more families receive at least eight sessions of 
family psychoeducation sessions from staff formally trained in family psychoeducation, 
code the component as ‘5’. 


e Ideally the sessions are provided during the first year. That is one of the reasons for selecting 
health records of patients who have been in the program for at least one year. If the total 
of eight sessions is only achieved in the second year this should be counted towards the 
rating. 


Comments: 
e Additional data sources to support comments: 
- Interviews with team leader, clinicians 
- Family psychoeducation manual, curriculum, or handouts 
e Specific information to include: 


- Are families offered group or individual psychoeducation? If they are not offered 
groups, are families connected to other families in any other way? Explain. 


- Is there a formal manual or curriculum that is followed? 


24. Cognitive Behavioral Therapy (CBT) 


Definition: Individual or group cognitive Behavioral therapy (CBT) delivered to individuals where indi- 
cated in the first year. CBT is an evidence-based treatment that is indicated for several clinical problems 
not limited to medication-resistant positive symptoms, anxiety, or depression. An appropriately trained 
professional should deliver CBT. 


The training may have been part of formal professional training or as continuing professional development 
(CPD). For both professional training and CPD the training should comprise both learning and supervision. 
Attending courses without supervision is not evidence of effective training. 


If provided as part of CPD training, the training should be from an established, authorized provider of CBT 
training (general CBT training as well as CBT for psychosis (CBT-P) are both acceptable). Recovery-oriented 
cognitive therapy (CT-R) for individuals with persistent schizophrenia is another validated CBT model. A 
formal certificate of competence provides confirmation of CPD training or training followed by supervision. 


Another way that CBT training and supervision can be confirmed is if the clinician has specific training 
in the use of a formal manual such as the RAISE Individual Resiliency Training (IRT) Manual. The IRT 
Manual provides a guide for CBT when the following modules have been delivered: Modules 5 (process- 
ing the psychotic episode), 6 and 14 (resiliencies), 8 (dealing with negative feelings), and 9 (coping with 
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symptoms). To meet criteria for this component, formal CBT should be provided for at least 10 sessions. 
Providing CBT informed-care or incorporating CBT principles into general case management is valuable 
(and could be acknowledged in the comments) but does not qualify for this component. Group formats for 
CBT have also been validated and can count towards this component. 


Rationale: CBT has been widely investigated and has been shown to be effective for symptom management 
and many other aspects of recovery. 


Component Scoring: 


e Data source to use for rating: Team leader and clinician interviews 


e Component response coding: If 60% or more of patients receive at least 10 sessions of CBT 
delivered by an appropriately trained professional, code the component as ‘5’. 


e Ideally the sessions are provided during the first year. That is one of the reasons for selecting 
health records of patients who have been in the program for at least one year. If the total of 
ten sessions is only achieved in the second year this should be counted towards the rating. 


Comments: 


e Additional data sources to support comments: 
- CBT curriculum/manual and materials 
e Specific information to include: 


- Where was CBT training received? Training can be received during professional 
training. It can also be obtained during continuing professional development. This 
is best confirmed by receipt of a diploma or professional recognition by a licensing 
authority which may be required for billing purposes. 


- Were any staff specifically trained in CBT for psychosis? 


25. Supporting Health 


Definition: Program takes steps to support patient health, including: 


1. 


2 
3 
4 
5. 
6 
7 


Refer to and engage with primary care 

Measure and record weight at least quarterly in the first year 

Provide feedback on weight gain and general advice on diet and exercise 
Monitor and document extrapyramidal side-effects 

Monitor glucose and or Haemoglobin 1a and triglycerides annually 
Monitor and document cigarette smoking habits annually 


Prescribe pharmacological supports to smokers wishing to quit 


a. Nicotine Replacement Therapy 
b.  Varenicline 
c. Buproprion 
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These steps must be taken with all patients unless otherwise specified. 


Rationale: Individuals with severe and persistent mental disorders such as schizophrenia have their lifespan 
shortened by at least a decade in part due to increased mortality from a range of physical illnesses. Early 
during illness, they have increased rates of smoking and weight gain, have poor exercise habits, and do not 
access primary health care. In addition, the determinants of health such as income, housing, and social 
support put them at increased risk of poor health outcomes. Connecting individuals to primary care, mon- 
itoring health indicators, and promoting smoking cessation can all impact health status. 


Component Scoring: 


e Data source to use for rating: Team leader, clinician, and prescriber interviews 


e Referral and engagement in primary care is rated as present if a team member records the 
details of the primary care provider in the health record or encourages and supports the 
patient to register with a primary health care provider. 


e Component response coding: If all seven items are present, code the component as ‘5. 


Comments: 
e Additional data sources to support comments: 


- Interviews with team leader and clinicians 


26. Annual Comprehensive Assessment 


Definition: All patients should receive an annual or ongoing comprehensive assessment, including: 
1. Educational involvement 
Occupational involvement 


Social functioning 


2 

3 

4. Symptoms 
5. Psychosocial needs 

6. Risk assessment of harm to self or others 
7 


Substance use 


Rationale: Individual patient circumstances change with time, and several diagnoses are time-dependent. 
Annual or ongoing assessments are considered a good clinical practice and provide the opportunity for 
updating treatment plans, communication with primary care providers, and ongoing planning. 


Component Scoring: 
e Data source to use for rating: Health record review 


e Component response coding: If 80% or more patients undergo annual or ongoing 
assessments that include at least six items above, code the component as ‘5’. 
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e ‘The elements of this assessment may be found in different areas of the health record. 
For example, weight/BMI and laboratory assessment are often found in the prescriber’s 
component of the health record. 


Comments: 


e Additional data sources to support comments: 


- Interviews with team leader and clinicians 


27. Services for Patients with Substance Use Disorders 


Definition: The FEP program offers the following: 


l. 


2 
3. 
4 


Assessment of substance use for all patients at intake and at annual review 

Addresses substance use in patient psychoeducation 

Addresses substance use in family psychoeducation 

Provides brief evidence-based psychotherapies such as motivational enhancement/ 
interviewing (ME/MI) or CBT for patients with substance use problems. Staff who provide 
these services must have had formal training and supervision in these therapies either during 
training or as Continuing Professional Development (CPD). This may include training on the 
specific modules used in the RAISE Individual Resilience Training. These are a few of several 
techniques that have been described as part of ME/MI: 

e Designate a change goal (i.e., substance use) 

e Ask related questions: What? How much? Benefits? Impact? Negatives? 

e Provide simple reflections 

e Provide complex reflections 

e Affirm 

e Summarize 

e Elicit/reinforce change talk 

e Collaborate with patient 

e Support self-efficacy 

e Roll with resistance 

e Address ambivalence 


Maintains continuity of care and patient engagement if patient is referred to specialized 
substance use services (e.g., detox, residential treatment). 


These criteria for FEPS substance use services meet criteria described as Dual Disorders 
Capability (Gotham, Brown, Comaty, McGovern, & Claus, 2013). 


Specialized services refer to addiction services such as detoxification, opiate treatment 
services such as outpatient maintenance programs, and residential addiction programs. 
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Alcoholics Anonymous or an outpatient addictions group do not count as a specialized 
service. 


Rationale: There is a high prevalence of substance use disorders in those with a first episode psychosis. The 
program should have staff trained to recognize and provide first-line interventions such as motivational 
enhancement. The training may have been part of their formal professional training or as continuing pro- 
fessional development (CPD). For both professional training and CPD, the training should comprise both 
learning and supervision. Attending courses without supervision is not evidence of effective training. 


Component Scoring: 


e Data source to use for rating: Interviews 


e Component response coding: If the first five items listed above under definition are present, 
code the component as ‘5’. 


Comments: 


e Additional data sources to support comments: 
- Interviews with the team leader, clinicians, and addiction/CBT specialist (if relevant) 


- Program policy and procedures, SUD program manual/materials 


28. Supported Employment (SE) 


Definition: Supported Employment (SE) or Individual Placement and Support (IPS) is provided to patients 
interested in gaining competitive employment. In contrast with mainstream employment services, SE is an 
evidence-based intervention targeted at individuals with mental illness and includes additional supports 
such as rapid job search, on the job support, negotiating workplace accommodations, and an emphasis on 
patient choice. The elements of SE include: 


1. 


Ps D E S 


Supported employment specialist is formally trained and has at least six months of experience 
as a supported employment specialist. 


Supported employment specialist is a member of the FEP team and attends team meetings. 


Supported employment specialist receives supervision at least two times per month from a 
trained supervisor. 


Ratio of caseload is 1:20 or less. 
Completes at least six employer contacts per week. 
Uses the career profile or an equivalent form. 


Tracks in-person employer contacts in a format that can be organized and searched. (Hand 
recorded in a personal notebook does not meet this criterion.) 


Rationale: Supported employment is an evidence-based program which increases employment rates. 
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Component Scoring: 
e Data source to use for rating: Interviews 


e This component assesses the individual components of IPS and rates the program on the 
number of elements provided to patients 


e Component response coding: If seven of more of the items listed above are present, code the 
component as ‘5’ 


NOTE: Identify percentage of time devoted to case management activities. If greater than 10%, remove that 
time from the supported employment specialist’s FTE in determining the 1:20 ratio. For example, the supported 
employment specialist is a full-time staff member. In the interview the supported employment specialist tells 
you they do case management activities 30% of the time. Because 30% is greater than the maximum allowable 
10%, reduce the employment specialist FTE from 1.0 to 0.7 when calculating the 1:20 ratio. 


Comments: 
e Additional data sources to support comments: 
- Interviews with the team manager and SE specialist 


- SE program description or materials 


29. Supported Education (SEd) 


Definition: Supported Education (SEd) is provided to patients who are interested in participating in edu- 
cation. The SEd provider should be integrated into the treatment team, assess educational potential, rec- 
ommend education that supports the goals of the individual, and link to appropriate local education and 
accommodation services. Evidence of supported education services include: 


FEPS program has a designated SEd specialist. 
SEd specialist is a FEPS team member and attends team meetings. 
SEd specialist has a caseload of at least three patients with education goals. 


SEd specialist uses a standardized form to document patient education goals. 


Gre “oe oe 


SEd specialist provides ongoing support including: 


Identification of educational programs 


a 
b. Identification of sources of financial aid 


c. Completion of applications and enrollment 
d. Management of coursework 
e. Acting as liaison between high school patient and high school teachers/counselors to 


develop disability/special education supports and transition plans 


Rationale: Supported education is a program which increases participation in education. 
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Component Scoring: 
e Data source to use for rating: Interviews 


e Component response coding: If SEd does not have a caseload of at least three patients, code 
component as ‘T’. If FEPS team meets items 1-4 above and at least four support sub items 
from item 5 ongoing support, code the component as ‘5’. 


Comments: 
e Additional data sources to support comments: 
- Interviews with the team leader and SEd specialist 


- SE program description or materials. 


30. Active Engagement and Retention 


Definition: Use of proactive outreach services (i.e., patient visits in the community) by clinicians to reduce 
missed appointments, engage patients with a first episode psychosis, and minimize drop-outs. 


Rationale: Active outreach promotes engagement and reduces drop-outs. 


Component Scoring: 


e Data source to use for rating: Team leader and clinician interviews 


e Component response coding: In the calculation only include percentage of out-in-the- 
community time by case managers/care coordinators. Average across case managers, but 
weight by their caseload. If more than 40% of all patient and family visits are out-of-office, 
code the component as ‘5’. 


Comments: 
e Additional data sources to support comments: 


- Program policies/procedures. For example, are they set up to do community 
visits? Policies that support outreach include financial support for transportation, 
insurance, and staff safety protocols. 


31. Patient Retention 


Definition: The dropout index is the ratio of the number of early dropouts (stayed in program <1 year) in the 
last 12 months to the total current caseload. This is a measure of all-cause discontinuation or the degree to 
which the program keeps individuals engaged in the program. 


Rationale: Patient and family engagement can be indirectly assessed through patient retention, as reflected 
in the numbers retained in the first year. 
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Component Scoring: 
e Data source to use for rating: Administrative data 


¢ Component response coding: The denominator is the number of FEPS patients in the 
program. The numerator is the number of patients who left the program in the last year 
before completing one year in the program. 


32. Crisis Intervention Services 


Definition: First Episode Psychosis program delivers crisis services or has links to crisis response services 
including crisis lines, mobile response teams, urgent care center, or hospital emergency rooms 


Rationale: Crises are a common occurrence in this population, and both organizational linkages and indi- 
vidual patients/patients care plans must reflect crisis plans. 


Component Scoring: 


e Data source to use for rating: Interviews 


e Component response coding: If the first episode psychosis team provides five-day a 
week, eight-hour per day drop-in services plus 24-hour phone crisis services, code the 
component as ‘5’. 


Comments: 
e Additional data sources to support comments: 
- Interviews with the program manager and case manager 


- Review of policy documents 


33. Communication between FEP and Inpatient Services 


Definition: If there is a hospitalization of a patient, the staff contacts the inpatient staff to be involved in 
treatment and discharge planning. Communication includes: 


1. Contact inpatient unit to establish communication plan. 
Visit with patient on inpatient unit. 
Communicate with family about admission. 


2 
3 
4. Involvement in discharge planning process. 
5. Receive/obtain hospital discharge summary. 
6 


Schedule outpatient appointment prior to discharge. 


Rationale: Close coordination between the treatment team and hospital staff is a hallmark of the assertive 
community treatment model, which has a strong evidence base demonstrating effectiveness in reducing 
hospital admissions. 
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Component Scoring: 


e Data source to use for rating: Health record review, Interviews 


e Component response coding: Health record review of the most recent five patients who 
have been admitted to hospital after starting the First Episode Psychosis program and 
discharged back to the care of the team. If the team performs all six communication 
activities when a patient is hospitalized, code the component as ‘5’. If the team has not had 
five admissions in the last year, records from earlier years could be used. If the number is 
still less than five, the evaluation could be made on three records. If there are fewer than 
three, the component cannot be rated. 


Comments: 
e Additional data sources to support comments: 
- Interviews with program manager and case manager 
- Program policy documentation review 
e Specific information to include: 


- What is the typical communication between first episode psychosis service staff and 
inpatient staff during hospitalization? 


- Are first episode psychosis staff typically involved in discharge planning? 


34. Timely Contact After Discharge from Hospital 


Definition: Patient in FEP service has face-to-face contact with FEP service provider within two weeks of 
discharge from hospital. 


Rationale: Post-discharge follow-up has been shown to reduce re-hospitalization. 


Component Scoring: 
e Data source to use for rating: Health record; Program documents. 


e Ifthe patient is referred to another clinical service after discharge from hospital, this is 
rated as a discharge and is not rated under this component. 


e Component response coding: If 80% or more of first episode psychosis patients admitted 
to hospital are seen at the first episode psychosis service for an outpatient appointment 
within 14 days of hospital discharge, code the component as ‘5’. 


Comments: 
e Additional data sources to support comments: 
- Interviews with program manager and case manager 
- Program policy documentation review 
¢ Specific information to include: 
- What is the typical communication between first episode psychosis service staff and 


inpatient staff during hospitalization? 
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35. Assuring Fidelity 


Definition: Program monitors quality using a fidelity scale or quality indicators linked to standards for 


program treatment components such as pharmacological and psychosocial treatments. 


Rationale: Adherence to the components identified in research as effective increases the chances of success- 
ful implementation. In general, higher fidelity is associated with improved outcomes, although this has yet 


to be demonstrated for the FEPS-FS 1.0. 


Component Scoring: 
e Data source to use for rating: Administrative Data; Interview with Team Leader. 


e The fidelity scale must have been published and previously used in fidelity assessments. 


e The quality indicators must be linked to discreet service components. For example, the 
proportion of new referrals receiving a telephone response within three working days and 
the proportion of new referrals being seen for an assessment within two weeks cannot be 
counted as two indicators. 


e Acceptable indicators include the following: 
- Percentage of referrals seen within two weeks 
- Percentage receiving Cognitive Behavioral Therapy 
- Percentage receiving Family Psychoeducation 
- Percentage receiving clozapine 
- Percentage receiving Supported Employment 
- Percentage receiving annual physical assessment 
- Percentage receiving motivational enhancement for substance use disorder 
- Any other indicators 


e Component response coding: If the program fidelity is assessed using a published fidelity 
scale by an external assessor, or if 11 or more discreet quality indicators are used to assess 
quality every two years, code the component as ‘5’. 


Comments: 
e Additional data sources to support comments: 


- Interviews with program manager. If the results of quality indicators or of a fidelity 
assessment are not part of the administrative data supplied by the program, ask the 
program manager. Ask about the measures used and ask for a copy of the most recent 
assessment. 


¢ Specific information to include: 


- The fidelity scale used or the quality indicators used and the method of collecting 
data for the quality indicators. 


- Health record review is the most reliable source of quality indicators. 
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APPENDIX A: PREPARING FOR A FIDELITY ASSESSMENT 


This document is a guide for site leaders at sites participating in the evaluation, outlining the steps required to 
prepare and complete the fidelity assessments. Please review carefully and keep in mind that preparations will 
need to begin several weeks in advance of the assessment date. 


The fidelity team will seek to schedule assessments at times convenient for the sites. It is important that the 
program information be provided in advance of interviews and that interviews occur over one or two days. 
The steps in the process are similar whether the assessment is onsite by the assessors or conducted remotely. 


1. The Fidelity Team Will Work with You 


Most of your contacts will be with the fidelity rater and the scheduler. These two roles may be filled by one 
person, but they will be the people who will work with you every step of the way to explain the process. 


2. Determine Primary Contact Person and Other Key Personnel 


The site leader should identify a primary contact person for the fidelity assessment (usually the site leader) 
to help organize and prepare for the fidelity assessment. The site leader should also assign a program docu- 
ment coordinator to assemble documents to upload to the data portal ahead of time and identify a health 
record abstractor who will be abstracting information from the health records. The health record abstractor 
should be an agency staff person who is not involved in clinical services (e.g., a quality assurance manager). 


The fidelity assessment will include interviews with at least four First Episode Psychosis (FEP) program staff: 
the team leader, the psychiatrist/prescriber, the supported employment/education specialist, and one case 
manager/therapist. The site leader will need to identify these four staff members for the interviews. Onsite 
fidelity assessments provide more flexibility to meet staff and patients. 


3. Establish Process and Obtain Approvals for Ethics/Privacy Oversight 


It is important that the fidelity reviews are conducted in an ethical manner that respects and protects the 
privacy of all patients, families, and staff. The fidelity assessor and site leader determine what processes and 
approvals are needed at your site. The process varies with the purpose of the fidelity assessment, specifically 
whether it is for research or quality assurance. The fidelity assessor will share with the site materials such 
as consent form templates/samples, confidentiality agreements (if your organization does not have its own 
form), and data collection tools. Please note that, depending on the required process at your program, this 
component may take up to several months to put in place. Once the required approvals have been obtained, 
please send a copy of the approval letter or email. 
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4. Schedule the Fidelity Assessment Dates 


The fidelity assessment includes interviews which require advance preparation by all participants to ensure 
they run smoothly. When scheduling your site interviews, it is important to consider the following factors: 


e The team leader should be interviewed first, with the remainder of the interviews 
scheduled over one or two days. 


e Schedule the assessment for days of the week when most or many of the program staff will 
be onsite (including the psychiatrist/prescriber). 


e Establish a timeline for expected ethics approval (if approvals have not yet been received). 


The fidelity assessment team will work with the primary site contact person and the fidelity rater to find a 
date that works for everyone. 


5. Prepare Materials for the Assessment 


As part of the assessment, the fidelity rater will review program materials and administrative data reports. 
The program document coordinator needs to prepare these materials in advance and provide them through 
the pre-planned mechanism at least one week prior to the assessment. 


NOTE: Only aggregate, de-identified data should be shared in advance. 

Do not include any documents containing personal health information. In the United States, review the 
following website for a list of what is considered personal health information: https://www.hipaa.com/hi- 
paa-protected-health-information-what-does-phi-include. Canada’s federal law, the Personal Information 
Protectionand Electronic Documents Act (PIPEDA) canbe foundathttps://www.priv.gc.ca/en/privacy-topics/ 
privacy-laws-in-canada/the-personal-information-protection-and-electronic-documents-act-pipeda/. 


The program document coordinator will capture components 9 and 10 in the table below by completing the 


Fidelity Program Data Table. No other information is required from the program document coordinator 
to capture this information. 
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Please provide the following: 


Document 


Description 


Program Brochure 


Includes description of services offered and patients 
served 


FTEs and Staff Roles 


List of the full-time equivalents for all prescribers and 
team members in the FEP program 


Program Admission Criteria 


Eligibility criteria for receiving FEP services 


Service Catchment Area 


Geographical area that the program services. If 
available, include estimated average travel time to 
the facility and specific geographic boundaries or 
populations within area. 


Patient Caseload Info 


Number of participants currently enrolled 

Numbers with diagnoses of: 

Schizophrenia spectrum disorder 

Major Depressive Disorder with psychotic features 
Bipolar Disorder with psychosis 

Attenuated Psychosis Syndrome or (Clinical High Risk) 
Non-Psychotic Disorders 


Assessment Templates 


Clinical and vocational assessment tools 
For example, include initial comprehensive assessment 
templates in the electronic health record. 


Psychoeducational Materials 


Manuals, curriculums, lesson plans, etc., used with 
patients/families for psychoeducation 


Hospital Discharge Info 


Number of days between hospital discharge and 
being seen in person by a member of the FEP team for 
each patient who has been hospitalized in the past 12 
months 


Targeted education to health/social 
services/community groups 


Number of information sessions delivered in the last 
12 months to first-contact people in health, education, 
community organizations, etc. The goal is to educate 
people to recognize signs of early psychosis and make 
them aware of the program. 


Clozapine Enrollment Information 


Number of patients with schizophrenia spectrum 
disorders who have been prescribed clozapine 


Patient Retention 


Number of first episode psychosis patients discharged 
in the last year who were in their first year in the 
program as a proportion of the total caseload of first 
episode psychosis patients 


Early Intervention 


Number of patients admitted to hospital prior to 
program entry in the last year as a proportion of the 
total number of patients admitted 


Results of most recent fidelity 
assessment or quality indicator 
assessment 
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6. Prepare Health Record Review Checklists 


As part of the fidelity review, the site will ensure that the fidelity rater receives a completed Health Record 
Review Checklist. This checklist is based on the health records of 10 patients who have been in the program 
for at least one year. The procedures are as follows: 


e The primary contact will identify and assign a number to the health records of active 
patients who have been in the program for at least one year. The primary contact will 
provide the fidelity assessment team with the total number of these health records. Using 
the total number of health records, the fidelity assessment team will provide the site with 
10 random numbers. They will then use these numbers to identify the health records. 


- For example, a site has 30 patients meeting the criteria. The site tells the assessment 
team that they have 30 health records. The research assistant identifies 10 
randomly selected numbers. The research assistant tells the primary contact to use, 
hypothetically, the 2nd, 7th, 8th, 10th, 11th, 15th, 19th, 22nd, 27th, and 30th health 
record for the review. 


e ‘The site will be responsible for cross-referencing these numbers to numbers assigned to the 
actual health record of patients. 


e Do not send health record IDs as this will violate privacy regulations. 


e Inthe case of a small program, it may be necessary to make accommodations in 
procedures. The assessment team will work with the primary contact person to identify 
the records to be reviewed. 


Logistics and privacy requirements for sharing health records should be determined during your ethics 
review/process. Things to think about include: 


e Will the health record abstractor need access to computers? 
e Will the abstractor need to be assigned a temporary password? 
e Will paper health records need to be requested? 


e Willa room need to be booked to review the health records? 


The health record abstractor will not record any identifying/personal health information in any notes that 
they take. 


42 First Episode Psychosis Services Fidelity Scale and Manual 


7. Plan Interview Schedule 


The interviews should be scheduled as close together as possible, preferably over one to two days. During the 


call, the fidelity rater will interview the following staff members: 


e The program manager/team leader (~1 hour 45 minutes) 
e A psychiatrist/prescriber (~1 hour) 

e A case manager (~1 hour 30 minutes) 

e A supported employment specialist (~30 minutes) 


e Other staff as required to collect adequate data for rating fidelity 


8. Confidentiality Forms and Other Permissions 


The fidelity rater may be required to sign a confidentiality form before starting the assessment. If this docu- 
ment needs to be signed in advance, please ensure it is sent to the fidelity rater to complete well in advance. 


Appendix A 
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APPENDIX B: HEALTH RECORD REVIEW 


Health Record Abstractor: 


Program: 


Date(s) of Health Record Review: 


Instructions: The health record review should be completed before the staff interviews. The abstractor 
should return the completed checklist to the Fidelity Rater. 


Health record selection. Two samples of health records should be selected. 


The first sample is selected according to the following criteria: 
1. 10 randomly chosen health records of 
2. patients with a Schizophrenia Spectrum Disorder 


3. who have attended the program for at least one year 


The second sample is selected according to the following criteria: 
1. Patients enrolled in program 


2. Last five who were admitted to hospital 


The health record review will require four hours or less for abstractors who are familiar with the record 
organization. The FEP program team leader should ensure the abstractor is familiar with the record organ- 
ization (e.g., through a meeting of the abstractor with a staff member who is familiar with the records and 
can orient the abstractor). 


The fidelity team will hold a one-hour training session with the site’s health record abstractor prior to the 
review and a one-hour debriefing after the completion of the review. 


The review of the first one or two health records may take longer to complete. After completing the review 
on two health records, the target time to complete each health record is expected to be no more than 15 
minutes. If the abstractor cannot locate the information for a component within five minutes, record that 
component as missing and give no credit. 

Components should not be rated according to a practitioner discipline; for example, the psychoeducation 
component includes all sessions or meetings regardless of which team member provides the psychoeducation. 


The fidelity assessment team will determine the scoring of some of the components, such as determining 
whether medication doses are in the acceptable range. The health record abstractor’s responsibility is to 
record health record information. 


To share comments about the rating of individual components or how to improve the manual, please write 
in the comments section at the bottom. 
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Health Record Review Checklist 


FEPS-FS-1.0 Response Possible Health Record # 

components format evidence 1 2 3 10 

4. Assigned Yes Health 

Case No Record 

Manager/Care initial 

Coordinator assessment 

14. Timely Number Health 

Contact with of days Record 

Referred from date 

Individual of referral 

to Ist ap- 
pointment 

15. Family Yes Assessment 

Involvement in No or notes 

Assessment 

16. Compre- Check Assessment 1 1 1 1 1 1 1 1 1 1 

hensive Clinical which of or progress 2 2 2 2 2 2 2 2 2 2 

Assessment 9 com- notes or 3 3 3 3 3 3 3 3 3 3 

- All 9 compo- ponents structured 4 4 4 4 4 4 4 4 4 4 

nents included complet- assessment 5 5 5 5 5 5 5 5 5 5 

in assessment® ed 6 6 6 6 6 6 6 6 6 6 
7 7 7 7 7 7 7 7 7 7 
8 8 8 8 8 8 8 8 8 8 
9 9 9 9 9 9 9 9 9 9 

17. Com- Check Assessment 1 1 1 1 1 1 1 1 1 1 

prehensive which of or progress 2 2 2 2 2 2 2 2 2 2 

Psychosocial 9 items notes or 3 3 3 3 3 3 3 3 3 3 

Needs Assess- complet- structured 4 4 4 4 4 4 4 4 4 4 

ment - 8 or ed assessment 5 5 5 5 5 5 5 5 5 5 

more compo- 6 6 6 6 6 6 6 6 6 6 

nents includ- 7 7 7 7 7 7 7 7 7 7 

ed in needs 8 8 8 8 8 8 8 8 8 8 

assessment? 9 9 9 9 9 9 9 9 9 9 

18. Clinical Yes 1: Plan 1 1 1 1 1 1 1 1 1 1 

Treatment No present 2 2 2 2 2 2 2 2 2 2 

Plan/Care Plan 2: Patient 

After Initial signs off on 

Assessment: plan 

addresses 

clinical and 

psychosocial 

needs. 

If treatment/ 

care plan 

present, mark 1 

at right. 

If patient signs 

off on plan, 

mark 2 at right. 
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FEPS-FS-1.0 Response Possible Health Record # 
components format evidence 1 2 3 4 5 8 9 10 
19. Yes Indicated 
Antipsychotic No in psychi- 
Medication atrist note 
Prescription or copy of 
prescription 
20. Antipsy- Yes Medication 
chotic Dosing No 
within Guide- R 
lines for Indi- Daily dose 
viduals with 
Psychosis - at 
6 months 
20. Second Yes Medication 
antipsychotic No 
medication 
Po at 6 Daily dose 
22. Patient Yes Patient 
Psychoeduca- No notes or 
tion: received group 
12 episodes by attendance 
trained clini- 
cian (individual 
or group) 
23. Family Yes 
Education No 
and Support: 
received 8 
sessions by 
trained clini- 
cian Cindivid- 
ual or group 
format) 
26. Annual Yes Annual as- 1 1 1 1 1 1 1 1 1 1 
Comprehen- No sessments 2 2 2 2 2 2 2 2 2 2 
sive Assess- on file OR 3 3 3 3 3 3 3 3 3 3 
ment: address- clear de- 4 4 4 4 4 4 4 4 4 4 
es 7 areas of scriptions 5 5 5 5 5 5 5 5 5 5 
functioning“ of review of 6 6 6 6 6 6 6 6 6 6 
1-7 com- 7 7 7 7 7 7 7 7 7 7 
ponents in 
progress 
notes 
NB: Components 33 and 34 are rated on the basis of data abstracted from health records of last five patients 
admitted to hospital after joining the program. 
33. Com- Yes Docu- 1 1 1 1 1 = 
munication No mentation 2 2 2 2 2 
between FEP supports 3 3 3 3 3 
and Inpatient each of the 4 4 4 4 4 
Services 6 items 5 5 5 5 5 
6 6 6 6 6 
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FEPS-FS-1.0 Response Possible Health Record # 


components format evidence 1 2 3 4 5 6 7 8 9 10 


34. Timely Yes 1 1 1 1 1 
Contact after No 
Discharge 

from Hospital: 
patient in 

FEP service 

seen for 
face-to-face 
contact with 

FEP service 
provider within 

two weeks of 
discharge from 
hospital 


i. Component 16 should include at least 8 of 9 items: (1) Time course of symptoms, change in 
functioning, and substance abuse (assessing whether there was change in functioning correlated 
with substance use changes or other changes); (2) Recent changes in behavior; (3) Risk 
assessment/harm to self or others; (4) Mental status exam; (5) Psychiatric history; (6) Premorbid 
functioning; (7) Co-morbid medical illness; (8) Co-morbid substance use; (9) Family history. 


ii. Component 17 should include all 9 items: (1) Housing; (2) Employment; (3) Education; (4) Social 
support; (5) Financial support; (6) Primary care access; (7) Family Support; (8) Past trauma; (9) 
Legal. 


iii. Component 26 should include all 7 areas: (1) Educational; (2) Occupational functioning; (3) Social 
function; (4) Symptoms; (5) Psychosocial needs; (6) Risk assessment of harm to self or others; (7) 
Substance use. 


iv. Component 33 should include all 6 items: (1) Contact inpatient unit to establish communication 
plan; (2) Visit with patient on inpatient unit; (3). Communicate with family about admission; (4) 
Involved in discharge planning process; (5) Receive/obtain a hospital discharge summary; and, (6) 
Schedule outpatient appointment prior to discharge. 
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FEPS-FS-1.0 components 
abstracted from Health Record 


Abstractor’s comments 


FEPS-FS-1.0 components 


4. Assigned Case Manager/Care 
Coordinator 


14. Timely Contact with Referred 
Individual 


15. Family Involvement in Assessment 


16. Comprehensive Clinical Assessment 
- All 9 items included in assessment 


17. Comprehensive Psychosocial Needs 
Assessment - All 9 items included in 
care plan 


18. Clinical Treatment Plan/Care Plan 
After Initial Assessment that addresses 
clinical and psychosocial needs. 


19. Antipsychotic Medication 
Prescription 


20. Antipsychotic Dosing within 
Recommendations for Individuals with 
Psychosis at 6 months 


22. Patient Psychoeducation 


23. Family Education and Support 


24. Cognitive Behavior Therapy 


26. Annual Comprehensive Assessment 


33. Communication between FEP and 
Inpatient Services 


34. Timely Contact after Discharge from 
Hospital 
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APPENDIX C: FIDELITY INTERVIEW GUIDE 


The First Episode Psychosis Services Fidelity Scale interview is a structured interview. The purpose of using 
a structured interview is to ensure that each interviewee is presented with the same questions in the same or- 
der. This ensures that the answers can be reliably aggregated and that programs can be compared with con- 
fidence. The interview guide is designed to be used in conjunction with the First Episode Psychosis Fidelity 
Scale (FEPS-FS-1.0) and the FEPS-FS 1.0 Manual. The interview is the primary source of information for 
the service components that describe clinician’s practice and team functioning. It is a secondary source 
of information for other services and is used to validate data obtained from health record review and ad- 
ministrative data. Once the required questions are asked, supplementary questions can be asked to clarify 
inconsistencies. 


The suggested questions are designed to cover all aspects of the fidelity assessment but should be supple- 
mented by any questions the rater may have based on the information submitted before the interview. All 
the questions should be asked to the team leader or other person taking the lead for the fidelity review. This 
person should be prepared to answer all the questions and be knowledgeable about all the information 
submitted prior to the fidelity interview. It is useful to provide team leader with the questions in advance. 


Who is interviewed? 
e The assessment team leader completes the whole interview. 


e The same questions can be used for the start of the more targeted interviews with other 
individuals, but more probing questions need to be asked to explore their perspective 
and role. 


e ‘The focus of the scale is services received by patients, and there is significant variability 
in the titles and professions titles of the person who delivers the specific service. The 
interviewer can identify the key providers during the orientation with the team leader. 


e The individual responsible for assessment, care-coordination, and therapy is interviewed 
on components 4, 15, 16, 17, 18, 22, 24, 26 27, 30, and 33. 


e The supported employment specialist is interviewed on components 28 and 29. 


e ‘The psychiatrist/prescriber is interviewed on components 5, 6, 8, 15, 16, 18, 19, 20, 21, 
and 25. 


Assessor Instructions: 


This interview guide is intended to be used with the team leader or program manager. Relevant questions 
may also be used with other staff (e.g., case managers/care coordinators, nurse, employment specialist, CBT 
specialist, peer support worker, etc.) as appropriate. If you know you will be interviewing other specialty roles, 
it is helpful to note in advance which questions are relevant. If you are using this interview guide with multiple 
individuals, you do not necessarily have to ask a question again once you are confident that you have made an 
accurate rating. 
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Most fidelity raters prefer to use a separate interview sheet for each interview. If you are doing multiple assess- 
ments in which the staff titles and roles are the same, you can tailor the interview sheet to fit the circumstances. 


If more than one person is completing the in-person interviews, the assessors should identify one team member 
to ask questions and another to take notes. The assessor leading the interview should view these questions as a 
guide and use their judgement as to which questions are appropriate to ask individual staff. 


Questions may be rephrased, omitted/added, or reordered as appropriate to gather the information needed to 
complete the ratings. 


Notes should not include any individual names or personal health information. Notes should pertain to pro- 
gram performance. Please avoid comments about performance of individual staff. 


Sample Introduction/consent: 


“Thank you for agreeing to be interviewed as part of the fidelity review. Before we begin, please review the 
consent form. I am happy to answer any questions. If you feel comfortable proceeding, please sign the bot- 
tom. Remember that you can always choose not to answer specific questions or stop the interview entirely.” 
(Give time for staff to review consent form and answer any questions.) 


Background: 


“Could you tell us about your role in the program, your professional background, and how long you have 
been involved in the program?” (This question can be used to help gauge which questions will be appropriate 
for this individual.) 
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Sample Questions: 


QUESTIONS RESPONSES 


Component 1: Practicing Team Leader 


e What are your academic and professional credentials? 
e Do you (the team leader) provide clinical services to patients? 


e How much time do you spend providing direct supervision and/ 
or clinical services? 


e Do you carry a caseload? Do you provide counseling or other 
direct services? 


e What Full Time Equivalent (FTE) do you devote to the FEP 
service? 


Component 2: Patient-to-Provider Ratio 


e What is the total FTE of all clinical staff assigned to the program, 
excluding the psychiatrist/prescriber? 


* How many patients are currently enrolled in the program? 


Component 3: Services Delivered by Team 


e What services are provided to the patients enrolled in your 
program? 


o Case Management Services including care coordination 


o Health Services (i.e., registering with primary care, measuring 
weight and metabolic monitoring, supporting smoking 
cessation) 


a Evidence-based psychotherapies such as CBT and ME/MI 


o Substance use management including, assessment, assessing 
readiness to change and ME/MI or CBT for addictions 


o Employment services such as Supported Employment or 
Supported Education 


o Family Education and Support 


o Patient Psychoeducation 


o Pharmacotherapy 


Component 4: Assigned Case Manager/Care Coordinator 


e What proportion of your patients get assigned a clinician who 
is responsible for delivering case management services or care 
coordinator upon enrollment in the FEP service? 


e Who provides the care coordination? 
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Component 5: Psychiatrist Caseload 


e Are patients assigned a psychiatrist/qualified prescriber 
(excluding family doctor) upon enrollment into the program? 


* How frequently can patients be seen by the psychiatrist/qualified 
prescriber (excluding family doctor) for urgent problems or to 
adjust medications early in treatment? 


e What is the caseload of the psychiatrist/qualified prescriber 
(excluding family doctor)? 


* How much FTE of psychiatrist/qualified prescriber time is 
available for that caseload? 


Component 6: Psychiatrist Role on Team 


e What is the role of the psychiatrist on the team? 
o Attend team meetings 
o See patients with another clinician 
o Accessible for consultation by team during the work week 


a Record medication, symptoms, side-effects, etc., in the same 
health record as the rest of the team 


o Registering with a primary care provider for those not already 
enrolled in primary care 


Component 7: Weekly Multi-Disciplinary Team Meetings 


e Does your program have clinical team meetings? 

e How often are team meetings held? 

e What issues are discussed during team meetings? 
o Case review, admissions, and discharges 
o Assessment and treatment planning and coordination 
o Discussion of complex cases 


e Who attends those meetings? 


Component 8: Explicit Diagnostic Admission Criteria 


e Does your program have explicit admission diagnostic criteria 
that are listed in a document? 


e If yes, what are your admission criteria? (Ask for a blank copy if 
you don’t have one already.) 


e How do you determine if a patient meets criteria? 


e Do you sometimes have exceptions? What the reasons for making 
an exception? 


e What percentage of your patients meet the program criteria after 
you have established a firm diagnosis ? 
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Component 9: Population Served 


Does your program serve a designated catchment area? 
How do you determine the size of the population that you serve? 


Does the program compare the expected incidence of first 
episode psychosis in the population with the annual admission 
rate to the program? 


What is the ratio of annual admission rate to the expected 
incidence? 


Component 10: Age Range Served 


What is the age range served by your program? 


Component 11: Duration of First Episode Psychosis (FEP) Program 


Is there a time limit on the length of time a person can receive 
services in your program? What is it? 


Component 12: Targeted Education to Health/Education/Social Service/Community Groups 


Do you provide education about psychosis and first episode 
psychosis services to first-contact professionals such as 
community mental health teams, school and college counsellors, 
social services agencies, family physicians, or emergency rooms? 


How many such presentations did you provide in the last year? 
Were any to services within your own agency? 

How often, where, and to whom? 

o family physicians 

o school and post-secondary counseling services 

o police service 

o hospital emergency rooms 

o youth social services agencies 


o community mental health services 


Component 13: Early Intervention 


How many patients do you have on your caseload with a first 
episode psychosis, not including any patients at clinical high risk? 


What proportion of these patients with a first episode psychosis 
received inpatient psychiatric care prior to starting or being 
admitted to first episode psychosis services? 
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Component 14: Timely Contact with Referred Individual 


e Do you have a time limit in which to respond to referrals? 


e What proportion of referrals are seen within two weeks at a 
face-to-face interview with a FEPS team clinician who can start 
treatment? 


Component 15: Family Involvement in Assessments 


e Do you have a policy of family involvement in the initial 
assessment? If yes, what is it? 


e Where is this policy document? Is it written or assumed? (Ask for 
a copy if you don’t have one already.) 


e How do you communicate the need and benefit of family 
involvement in the initial assessment prior to the first meeting? 


Component 16: Comprehensive Clinical Assessment at Enrollment (all 9 components completed) 


e Please outline what information is collected at the initial 
assessment upon enrollment into the FEP Service: 


o Change in functioning from stable baseline 
o Time of onset of substance use and frequency of use 
o Recent changes in behavior 
o Risk assessment/harm to self or others 
a Mental status exam 
o Psychiatric history 
o Premorbid functioning 
o Co-morbid medical illness and substance use 
o Family history of mental disorder 
e Who reviews the initial assessment after completion? 


e Where would we find it in the health record? (Ask for a blank 
copy if you don’t have one already.) 
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Component 17: Comprehensive Psychosocial Needs Assessment 


e Do you assess the psychosocial needs of your patients? 
e Which of the following areas are assessed? 

o Housing 

o Employment 

o Education 

o Social support 

o Past trauma 

o Family physician 

o Legal 

o Financial support 


o Family support 


Component 18: Clinical Treatment/Care Plan after Initial Assessment 


e Do your patients have clinical treatment plans? 
e How do you engage patients in developing the plans? 


e How do you document patient involvement in developing the 
plans? 


o Do the patients sign off on the plan? 
e What is addressed in the clinical treatment plan? 
a Needs 
a Goals 
o Preferences 
o Pharmacotherapy 
o Psychotherapy 


o Substance Use 


o Mood problems 
o Suicide prevention 


o Weight management 


Component 19: Antipsychotic Medication Prescription 


e What percentage of patients with a confirmed first episode 
schizophrenia spectrum disorder are offered antipsychotic 
medications? 


Component 20: Antipsychotic Dosing within Recommendations for Individuals with Psychosis 


e What is your policy regarding antipsychotic dosing? 


e What proportion of antipsychotic prescriptions are within the 
recommended government approved dose range (at 6 months)? 


e What percentage of patients are on dosages below the guidelines 
at 6 months? 
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Component 21: Clozapine for Medication-Resistant Symptoms 


e Does the program use clozapine when indicated? 


e How many patients with schizophrenia spectrum diagnosis do 
you have in the program? 


e How many of your patients with a schizophrenia spectrum 
diagnosis are on clozapine? 


Component 22: Patient Psychoeducation 


e Are patients provided with information about the schizophrenia 
spectrum disorders? If yes, how? 


e What languages do you provide written materials in? 


e Do you have a structured psychoeducational curriculum 
(e.g., IIIhess Management and Recovery curriculum, Wellness, 
Recovery Action Plan (WRAP) training)? If yes, what is it? (Ask 
for a copy if you don’t have one already.) 


e Who delivers the curriculum? 


e Do the staff who deliver Psychoeducation (PE) have formal 
training in PE? If yes, what is it? 


o A formal part of their professional training including: 
o Training 
o Supervision 
a Continuing professional development including: 
o Training 
o Supervision 


e How is this curriculum delivered? Are patients engaged in formal 
group or individual psychoeducational sessions? What are the 
topics of discussion? 


o Developing coping and self-help strategies 
o Developing resiliency 

o Dealing with the symptoms of psychosis 

o Activities of daily living 

o Educational/academic supports 

a Vocational/employment supports 

o Housing supports 

o Substance use 

o Establishing social relationships or connections 
o Peer support 

o Income support 


e Do you document whether a patient receives this curriculum? If 
yes, how do you document this? 
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Component 23: Family Education and Support 


e How do you deliver information about psychosis to family and 
caregivers? 


e Do you provide written materials in various languages? 
e Are families engaged in individual or group sessions? 


e Do you document family and caregiver participation in your 
sessions? 


e Who delivers the Family Psychoeducation? 


e Do the staff who deliver Family Psychoeducation have formal 
training in Family Education and Support? If yes, what is it? 


o A formal part of their professional training including 
o Training 
o Supervision 
o Continuing professional development including 
o Training 
o Supervision 


e Do the staff who deliver Family Psychoeducation receive 
supervision? Describe the frequency and intensity of the 
supervision. 


e Which of the following topics are addressed? 
o Information about psychosis 
o Information about medications 
o Information about coping with stress 
o Recognition and prevention of relapse 
o Collaboration with clinicians 


o Effective communication 


o Supporting recovery 
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Component 24: Cognitive Behavioral Therapy (CBT) 


Does the program offer Cognitive Behaviour Therapy (CBT) for 
patients? 


Who delivers CBT? 


Do the staff who deliver CBT have formal training in CBT? If yes, 
what is it? 


o A formal part of their professional training including: 
o Training 
o Supervision 
o Continuing professional development including: 
o Training 
o Supervision 
What are the indications for which CBT is provided? 
o Anxiety or depression 
o Residual or medication-resistant positive symptoms 
o Substance Use Disorders 
How many sessions are typically given to an individual patient? 


What proportion of patients receive at least 10 sessions of CBT in 
their first year in the program? 


Where is CBT documented? 


Do you offer CBT in individual or group formats or both? 
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Component 25: Supporting Health 


e Primary care linkage: 


O 


Do you record whether your patients have a primary medical 
care provider? 


Do you refer patients or support patient application to a 
primary care provider? 


Where do you document support for primary care 
engagement? 


e Non-pharmacological weight management: 


O 


O 


O 


O 


Do you routinely monitor weight? 
How often do you monitor weight? 
Where is this documented? 


Does someone on the team provide feedback to patients on 
their weight? 


Does someone on the team provide advice and suggestions 
about diet and exercise? 


e Monitor glucose and triglycerides annually: 


O 


O 


O 


O 


Do you routinely assess blood glucose and or haemoglobin 
AIC? 


Do you routinely assess triglycerides? 


Who on the team is responsible for ensuring that these are 
done? 


Where is this recorded? 


e Smoking reduction: 


O 


O 


O 


[0] 


Component 26. Annual Comprehensive Assessment (all 7 components) 


Do you routinely assess patients’ smoking habits? 

Who is responsible for assessing and recording this? 

Do you prescribe pharmacological aids to smoking cessation? 
o Nicotine Replacement Therapy 
o Varenicline 
o Bupropion 


Where is this recorded? 


e Do you conduct an annual comprehensive assessment? 


e Please outline what information is collected at the annual 


assessment: 

o Educational functioning 

o Occupational functioning 

o Social Functioning 

o Symptoms 

o Psychosocial needs 

o Risk assessment of harm to self/others 
o Substance use 


e Where would we find it in the health record? (Ask for a blank 
copy if you don’t have one already.) 
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Component 27: Services for Patients with Substance Use Disorders 


* Does the service exclude patients with a substance use disorders? 


. f patients with substance use disorders are accepted, how do you 
assess for the presence of substance use? 


* Do you address substance use in patient psychoeducation? 
. Do you address substance use in family education? 


* Does the program provide brief interventions such as motivational 
enhancement/motivational interviewing of CBT for substance use 
problems? 


* Does the program refer patients with moderate to severe (i.e., DSM-5 
moderate 4-5 symptoms, severe 6 or more symptoms) substance use 
disorders to specialized addictions services ranging from detoxifica- 
tion to residential treatment? If the patient is attending specialized 
services does the service maintain them in the FEP program? Liaise 
with the addiction service and maintain contact with the patient? 


e Who delivers the Motivational Enhancement (ME) or Motivational 
Interviewing (MI)? 


* Do the staff who deliver ME/MI have formal training in ME/MI? If yes, 
what is it? 


o A formal part of their professional training including: 
o Training 
o Supervision 
a Continuing professional development including: 
o Training 
o Supervision 
* How many sessions are typically given to an individual patient? 


Evidence suggests that only training and supervision ensure that clinicians 
use correct ME/MI strategies in practice. If there is uncertainty about the MI/ 
ME delivered ask an open question about which techniques are used. If the 
respondent does not understand the question, use one or two of the follow- 
ing as an example: 


« What techniques or skills do you use in motivational interviewing? 
Can you give examples from a recent case? 


* These area few of several techniques that have been described as 
part of MI: 


* Designate a change goal (i.e. reduce substance use) 


« Ask related questions, including: What? How much? Benefits? 
Impact? Negatives? 


« Provide simple reflections 

« Provide complex reflections 
e Affirm 

* Summarize 

e  Elicit/reinforce change talk 
* Collaborate with patient 

* Support Self-Efficacy 

e Roll with resistance 

e Address ambivalence 


* Do the staff who deliver ME/MI or CBT for psychosis have formal 
training in those modalities? If yes, what is it? 


o A formal part of their professional training 


o Continuing professional development with a formal confirmation 
of proficiency 


o Continuing professional development without confirmation of 
proficiency 


« What other services are offered to address co-occurring substance 
use disorders? 
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Component 28: Supported Employment (SE) 


e Do you have a separate role on your team for SE specialist (or is 
this part of the job responsibilities for a care manager)? 


e What training has your SE specialist received? (Prompts: IPS 
Employment Center online course, IPS practice manual, SAMHSA 
toolkit, RAISE toolkit, On Track) 


e What supported employment model do you follow? (Prompts: 
IPS, SAMHSA toolkit, RAISE NAVIGATE model, On Track) 


e How much experience does your SE specialist have? Is it >6 
months? 


e Is SE specialist a team member who attends team meetings? Is it 
at least twice monthly? 


e Does your SE specialist receive supervision from an experienced 
SE supervisor? (What frequency?) 


e How many patients are on each SE specialist’s caseload? 


e How many employer contacts doe your SE have? Is it tracked on 
a specific form? Does SE have =6 employer contacts per week? 


e Does your SE specialist use a Career Profile or equivalent? 


e Does your SE specialist track in-person employer contacts? 


Component 29: Supported Education (SEd) 


e Do you offer supported education services to all patients 
interested in education? 


e Who provides supported education services? 


e What training has your SEd specialist received? (Prompts: IPS, 
SAMHSA toolkit, RAISE model, On Track) 


e Is your SEd specialist a member of FEPS team? 
e Do you offer SEd services to all patients interested in education? 
e Who provides SEd services? 


e What training has your SEd specialist received? (Prompts: IPS, 
SAMHSA toolkit, RAISE model, On Track) 


e Is your SEd specialist a member of FEPS team? 
e What is your SEd’s caseload of patients with education goals? 


e Does your SEd specialist complete and documents educational 
goals? What forms do they use for documenting goals? 


e Does your SEd specialist explore education programs with 
interested patients? 


e Does your SEd support educational financial planning (e.g., 
loan forgiveness, financial aid, loan repayment, scholarship 
application)? 


e Does your SEd specialist support patients in applying to 
programs? 


e Does your SEd specialist help students manage course work? 


e Does your SEd specialist identify legislated support for high 
school students? 
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Component 30: Active Engagement and Retention 


* Do you havea policy about where case managers/care 
coordinators meet their patients and families? If yes, what is it? 


e Where is this policy documented? Is it written or assumed? 


e What proportion of case managers/care coordinator visits are 
out of the clinic and in the community? 


Component 31: Patient Retention 


For this component we are considering “all causes discontinuation” as 


the measure of retention. This means all patients who leave the program 


before the end of their first year in the program. 
Questions: 


e What is the total caseload of patients with a first episode 
psychosis in your program at this time? 


* How many patients were terminated/discharged from your 
program in the last 12 months? 


* Of those who were terminated/discharged, how many were 
active patients for less than 12 months? 


e Count the number of early terminators and divide by caseload size. 


Component 32: Crisis Intervention Services 


e How does your program manage patient-related crises during 
and after hours? 


o Formal linkages to out-of-hours services 
o Respond to crisis calls during office hours 
o Drop-in crisis visits during office hours 


o 24-hour phone and in-person crisis services 


Component 33: Communication Between FEP and Inpatient Serv 


ices 


e How many patients on the caseload were admitted to an 
inpatient psychiatric unit in the last year? 


e What arrangements do you have in place to ensure 
communication with inpatients and continuity of care during and 
after hospitalization? 


e Which of the following roles are part of FEPS staffs’ standard 
communication protocol for patients admitted to hospital? 


o Contact inpatient unit to establish communication plan 
o Visit with patient on inpatient unit 

o Communicate with family about admission 

o Involvement in discharge planning process 

o Schedule outpatient appointment prior to discharge 


o Obtain a hospital discharge summary 
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Component 34: Timely Contact after Discharge from Hospital 


e Do you have a target for how soon patients who have been 
admitted to hospital should be seen after discharge? 


e How soon were each of the last five patients admitted to hospital 
seen after discharge? 


Component 35: Assuring Fidelity 


e Does the program use a set of quality standards? 
e Who set or published these standards? 
e Do you assess program quality with a Fidelity Scale? 
o Which fidelity scale do you use? 
o Has the fidelity scale been published? 
o Is the fidelity assessment done by an independent rater? 
e Do you measure quality indicators for 
o Percentage of referrals seen within 2 weeks 
o Percentage receiving Cognitive Behavioral Therapy 
o Percentage receiving Family Psychoeducation 
o Percentage receiving clozapine 
o Percentage receiving Supported Employment 


o Percentage receiving annual physical assessment 


o Percentage receiving motivational enhancement for 
substance use disorder 


o Any other indicators 
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APPENDIX D: SCORING SHEET 


Study Name: Today’s date: 
Site name: Fidelity interview dates: 
Fidelity Round: Year 1 Year 2 
Item Score 


= 


Practicing team leader: 


2 Participant/provider ratio: 

3 Services provided by team: 

4 Assignment of case manager/care coordinator: 
5 Psychiatrist caseload: 

6 Psychiatrist role on team: 

T: Weekly multi-disciplinary team meetings: 

8 Explicit diagnostic admission criteria: 

9 Population served: 

10 Age range served: 

1 Duration of FEP program: 

12 Targeted Education to health/social service/community groups: 
13 Early Intervention: 

14 Timely contact with referred individual: 

15 Family involvement in assessments: 

16 Comprehensive clinical assessment: 

i7 Comprehensive psychosocial needs assessment: 
18 Treatment/Care Plan after initial assessment: 

19 Antipsychotic medication prescription: 

20 Antipsychotic dosing within recommendations: 
21 Clozapine for medication-resistant symptoms: 
22 Patient psychoeducation: 

23 Family education and support: 

24 Cognitive behavioral therapy (CBT): 

25 Supporting Health: 

26 Annual comprehensive assessment: 

27 Services for patients with Substance Use Disorders: 
28 Supported employment (SE): 

29 Supported education (SEd): 

30 Active engagement and retention: 

31 Patient Retention: 

32 Crisis intervention services: 

33 Communication between FEP and inpatient services: 
34 Timely contact after discharge from hospital: 

35 Assuring fidelity: 


Total Score: 
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APPENDIX E: FEEDBACK REPORT TEMPLATE 


Program Name: 

Program location: 

Date(s) of fidelity review: 
Assessment type - Remote/Site Visit: 
Assessor(s): 


Date of Report: XX-XX-XXXX 


Introduction 


This report includes the findings from fidelity assessment using the First Episode Psychosis Fidelity Scale at 
XXX program. Measuring fidelity, or adherence to the model, is important to ensure that the services deliv- 
ered are consistent and are of high quality. This fidelity assessment was conducted using the First-Episode 
Psychosis Services Fidelity Scale (FEPS-FS-1.0). 


The purpose of fidelity assessments is to better understand the services delivered in First Episode Psychosis 
Services programs. It is expected that ratings will vary widely depending on available resources. A rating in 
the range of 1 to 3 indicates a need for improvement. A score of 4 is considered good and 5 exemplary. The 
nature and detail of the report will vary depending on whether there has been a site visit or a remote assess- 
ment. It will also vary according to the purpose of the fidelity assessment and the nature of the relationship 
between the assessor and the program. 


A fidelity assessment in which the primary purpose is for quality improvement should be more detailed 
and specific to the challenges faced by a program. Whatever the purpose of the assessment this report is an 
opportunity to reflect and identify areas of strength as well as opportunities for quality improvement. This 
template gives some examples based on composite anonymized reviews. 


Summary 


Program XXX XXXX located in XXX XXXxX, serves the people of YYY region. Clinical team members in- 
clude X team leaders, N care coordinators/case managers, N therapists, N supported employment specialists, 
and N psychiatrists (Other team members should be included, such as nurses, peer support workers, etc.). 
The team currently serves N patients with a diagnosis of a schizophrenia spectrum disorder. The team fol- 
lows the XXX XXXX model and XXX model or manual for delivering patient and family psychoeducation. 
The team delivers Cognitive Behaviour Therapy using XX model and uses motivational interviewing for 
patients who abuse substances. 


On dates XXXX and XXXX we conducted a remote/on site fidelity review which consisted of interviews 
with the (Titles). The assessor reviewed the health record checklist—review of 10 patient records who had 
been in the program at least one year and 5 who had had an admission to hospital in the last year, both 
prepared by a non-clinical staff member at the site, and program documents. 
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The program achieved either good or high fidelity (a score of £ and ‘5’, respectively) across the majority (XX 
%) of components on the FEPS-FS-1.0 scale. The program reports that their clinicians have received formal 
training and supervision in CBT (Name model), Patient Psychoeducation (Name model), Family Education 
(Name model) and Motivational Interviewing, Supported Employment (Name model) and Supported 
Education (Name model). The program provides (list services delivered or not delivered) Pharmacotherapy, 
CBT, Patient and Family Psychoeducation, Supported Employment and Supported Education services to 
support health and deal with Substance Use Disorders. We applaud the team for (Name a special achieve- 
ment of the program if appropriate). Suggest a quality improvement initiative that is supported by the re- 
view. For example, “The program has a high patient-to-staff ratio, which offers the team the opportunity to 
provide services to more patients.” 


Fidelity components and service delivery areas that the agency may wish to prioritize for improvement 
include: (Focus on the components that score 3 or less.) 


Component XX: YYY YYY 
Component XX: YYY YYY 
Component XX: YYY YYY 


For a step-by-step guide on how to improve in these areas, see ratings and suggestions below. 


Ratings per component 
The following section provides the score for each of the 35 fidelity components from the First-Episode 
Psychosis Services Fidelity Scale (FEPS-FS-1.0). 


The following information will be provided per component: 


Score 
Each component will receive a score between 1 (indicating a low level of fidelity) to 5 (indicating a high level 
of fidelity). A rating of 4 is considered good fidelity to the model. 


Data source used for rating 

Specification of the data source that the assessor used to determine the rating for that component. Additional 
data sources helped to inform the narrative feedback, but this section identifies the data sources used for the 
rating. 


Program practice and strengths 
How the program delivers/adheres to this component. 


Quality improvement opportunities 


May include suggestions for quality improvement opportunities related to this component. 


Additional comments 
Additional comments the assessor may wish to include (e.g., relevant contextual/scoring information). 
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Example of Completed Feedback Form 


RATING 


COMMENTS 


1. Practicing Team Leader 


Score 


Data source used for rating: 
Team leader interview 


Program practice and strengths: 
Team leader carries administrative, clinical, and supervision duties. 


Quality improvement opportunities: 
No comment 


2. Patient-to-Provider Ratio 


Data source used for rating: 


acare Team leader interview and program documents 
Program practice and strengths: 
i Ratio = 4.2 patients per 1 staff FTE 


Quality improvement opportunities: 
No comment 


3. Services Delivered by Team 


Data source used for rating: 


ACOG All team member interviews 
Program practice and strengths: 
Team provides: 1. Case management/care coordination; 2. 
Psychotherapy; 3. Substance use management; 4. Supported 
employment; 5. Family education/support; 6. Patient 

A psychoeducation; 7. Pharmacotherapy 


Quality improvement opportunities: 

Missing provision of health services. Consider including a nursing 
staff as a member of the FEP team. We recommend patients are 
routinely weighed at each visit with the psychiatrist and have their 
smoking habits monitored annually. Nursing staff can deliver these 
services, which will also reduce burden on other staff members. 


4. Assigned Case Manager/Care Coordi 


nator 


Data source used for rating: 


Score Health record review 
Program practice and strengths: 
5 100% of patients are assigned a case manager or receive care 


coordination services when they enroll. 


Quality improvement opportunities: 
No comment 


Appendix E 


7I 


RATING 


COMMENTS 


5. Psychiatrist Caseload 


Score 


Data source used for rating: 
Psychiatrist interview and program documents 


Program practice and strengths: 
Ratio = 10 patients per 0.2 psychiatrist FTE 


Quality improvement opportunities: 
No comment 


6. Psychiatrist Role on Team 


Data source used for rating: 


aes Team leader and psychiatrist interview 

Program practice and strengths: 

Psychiatrist attends only half the team meetings, sees patients 
7 with another clinician, is accessible by the team during the work 


week for consultation, and shares health record with other team 
members. 


Quality improvement opportunities: 
No comment 


7. Weekly Multi-Disciplinary Team Meet 


ings 


Data source used for rating: 


SCOre Team leader interview 
Program practice and strengths: 
5 All team members attend weekly team meetings. 


Quality improvement opportunities: 
No comment 


8. Explicit Diagnostic Admission Criteria 


Data source used for rating: 


Score Team leader interview and program documents 
Program practice and strengths: 
5 All patients meet program’s eligibility guidelines. 


Quality improvement opportunities: 
No comment 


9. Population Served 


Data source used for rating: 


Score Team leader interview, program documents, census.gov 
Program practice and strengths: 
Over the last 12 months, the program enrolled 7% (N = 13) of the 
expected incidence of first episode psychosis in region (N = 184). 

1 Quality improvement opportunities: 
Change in the number of patients served by the program is an 
administrative and funding issue rather than a clinical practice 
issue under the control of clinicians. 
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RATING COMMENTS 


10. Age Range Served 


Data source used for rating: 


Score i ; 
Score Team leader interview, program documents 


Program practices and strengths; 
The program offers services to patients in age range of 18 to 30. 
This represents 25% of the age range of onset of 14 - 65. 


Quality improvement opportunities: Change in the program age 
range mandate is an administrative and funding issue rather than a 
clinical practice issue under the control of clinicians. 


11. Duration of First Episode Psychosis (FEP) Program 


Data source used for rating: 


Score 7 À 
Team leader interview 


Program practice and strengths: 
The mandated length of the program is two years. 


Quality improvement opportunities: 

3 It is important for a program to have a clear duration to ensure 
appropriate access and for new patients. The risk of relapse 

is highest in the first years. Some studies have found better 
outcomes for those patients who are provided first episode 
psychosis services for at least 3 years. 


12. Targeted Education to Health/Education/Social Service/Community groups 


Data source used for rating: 


Score Team leader interview and health record review 
Program practice and strengths: 
Over the past 12 months, the program delivered 10 face-to face- 
information sessions to first-contact professionals including 
schools, mental health community providers, and a physician. 

4 Quality improvement opportunities: 


Aim for at least 12 in-person information sessions per year. 
Additional comments: 

Though podcasts do not meet criteria for this component, we 
applaud the team using podcast technology, which has the 
potential to deliver information to a wide audience. 


13. Early Intervention 


Data source used for rating: 


Score ; 
Program documents and an email 


Program practice and strengths: 
Of those currently enrolled in the program (N = 25), 88% (n = 22) 
were hospitalized prior to enrollment. 


Quality improvement opportunities: The following strategies have 
been found to be helpful for earlier intervention. Outreach to local 
community first contacts, services adequate to meet the expected 
incidence. 
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RATING 


COMMENTS 


14. Timely Contact with Referred Individual 


Data source used for rating: 


BeOtS Health record review 
Program practice and strengths: 
Z Patients were seen in person within 14 days after referral in 9 out of 


10 patient records reviewed. 


Quality improvement opportunities: 
No comment 


15. Family Involvement in Assessments 


Data source used for rating: 


score Health record review 

Program practice and strengths: 

9 families of the 10 patient records reviewed were involved in the 
5 initial assessment or seen within the first month of the patient’s 


initial assessment. 


16. Comprehensive Clinical Assessment 


Quality improvement opportunities: 
No comment 


Data source used for rating: 


Score Health record review and interviews 
Program practice and strengths: 
P Program consistently assesses all 9 key clinical areas during 


enrollment. 


Quality improvement opportunities: 
No comment 


17. Comprehensive Psychosocial Needs 


Assessment 


Data source used for rating: 


Score Health record review and interviews 

Program practice and strengths: 

Program consistently assesses 8 of 9 psychosocial areas during 
5 enrollment in the 10 patient records reviewed. 


Quality improvement opportunities: 
Ensure to document the patient’s financial situation (e.g., receiving 
benefits or disability). 


18. Clinical Treatment Plan/Care Plan af 


ter Initial Assessment 


Score 


Data source used for rating: 
Health record review 


Program practice and strengths: 
Treatment plan found for 8 of 10 patients. 


Quality improvement opportunities: 

Ensure that all patients have a treatment/care plan and that all 
patients sign the plan as a form of documentation that they agree 
to the plan. 
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RATING 


COMMENTS 


19. Antipsychotic Medication Prescription 


Data source used for rating: 


score Health record review 
Program practice and strengths: 
é 9 out of 10 patients reviewed were prescribed an antipsychotic 


medication upon confirmation of diagnosis. 


Quality improvement opportunities: 
No comment 


20. Antipsychotic Dosing within Recom 


mendations for Individuals with FEP 


Data source used for rating: 


oe Health record review 
Program practice and strengths: 
6 out of 10 patients reviewed received medication dosages in line 
with US Federal Drug Administration approved dose ranges for 
j antipsychotic medications for schizophrenia spectrum disorders. 


Quality improvement opportunities: 

Aim for doses in the range shown to be effective. Research shows 
that higher than recommended doses or multiple antipsychotics 
cause more side-effects with no incremental benefit. Clozapine is 
an option for treatment resistant positive symptoms. 


21. Clozapine for Medication-Resistant Symptoms 


Data source used for rating: 


3COrG Psychiatrist interview and program documents 
Program practice and strengths: 
5 12% (n = 3) of patients are prescribed clozapine. 


Quality improvement opportunities: 
No comment 


22. Patient Psychoeducation 


Data source used for rating: 


gore Health record review 
Program practice and strengths: 
Š Program delivers patient psychoeducation to all patients by 


trained clinicians. 


Quality improvement opportunities: 
No comment 


23. Family Education and Support 


Data source used for rating: 


Score Health record review 

Program practice and strengths: 

Families from 4 out of 10 patients received 8 sessions of family 
> psychoeducation. 


Quality improvement opportunities: 

Continue efforts to engage families in the patient’s treatment. 
Family psychoeducation can be delivered in-person, by telephone, 
or by video-conference. 
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RATING 


COMMENTS 


24. Cognitive Behavioral Therapy (CBT) 


Score 


Data source used for rating: 
Primary clinician interview 


Program practice and strengths: 
Clinicians are trained in CBT but only 20% of patients have 
received 10 sessions of CBT. 


25. Supporting Health 


Score 


Quality improvement opportunities: Continue to engage patients 
and provide psychoeducation on the benefits of CBT for specific 
problems. Look at program flexibility to deliver services for 
patients at times and places that work for patients. 


Data source used for rating: 
Team leader and Psychiatrist interview 


Program practice and strengths: 

Program performs these functions: 1. Provides feedback on weight 
gain and general advice on diet and exercise; 2. Monitors and 
documents extrapyramidal side-effects; 3. Refers/engages with 
primary care. 


Quality improvement opportunities: 

Weigh patients at least quarterly in the first year after enrollment. 
Monitor all patients’ glucose levels, triglycerides, and smoking 
habits at least annually. For those who smoke tobacco and are 
interested in quitting, offer pharmacological interventions to 
support this goal. Clarify process and accountability for delivery of 
these services. 


26. Annual Comprehensive Assessment 


Data source used for rating: 


Score Health record review 
Program practice and strengths: 
, Program routinely assessed 6 of 7 key clinical and psychosocial 


areas at least annually. 


Quality improvement opportunities: 
Include an assessment of metabolic parameters in the assessment. 


27. Services for Patients with Substance Use Disorders 


Data source used for rating: 


Score ; et ah 2 
Primary clinician interview 
Program practice and strengths: 
5 Team maintains high level of engagement with patients receiving 


specialized substance use services. 


Quality improvement opportunities: 
No comment 
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RATING COMMENTS 


28. Supported Employment (SE) 


Data source used for rating: 


moe Supported employment specialist and team leader interviews 


Program practice and strengths: 
SE specialist is well trained. 


Quality improvement opportunities: 

3 SE should aim for at least 6 employer contacts per week for 
patients looking for employment and track these contacts in 

a format that can be easily searched and organized (e.g., ona 
spreadsheet). The fewer contacts made were likely due to only two 
patients searching for employment. 


29. Supported Education (SEd) 


Data source used for rating: 


Score Supported employment specialist interview 


Program practice, strengths, and challenges: 

Supports patients looking for college programs (connects with 
university program coordinators, attends school tours, and 

5 supports application completion) and those still in high school 
(attends IEP meetings and ensures accommodations are met). 


Quality improvement opportunities: 
No comment 


30. Active Engagement and Retention 


Data source used for rating: 


Peo Team leader and clinician interview 

Program practice and strengths: 

Clinicians spend on average 14.75% out in the community. 
2 Quality improvement opportunities: 


Increasing time in the community will improve patient engagement 
in the program. Aim for an average of at least 40% of time out in 
the community for those clinicians providing case management 
and care coordination services. 


31. Patient Retention 


Data source used for rating: 


pore Program documents 
Program practice and strengths: 
4 Dropout index = 0.20 


Quality improvement opportunities: 
Spending more time in outreach may reduce early dropout rate. 
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RATING 


COMMENTS 


32. Crisis Intervention Services 


Data source used for rating: 


ote Team leader and primary clinician interviews 
Program practice and strengths: 
4 Program is very accessible to patients in crisis both via phone and 


in-person 


Quality improvement opportunities: None 


33. Communication Between FEP and Inpatient Services 


Data source used for rating: 


scare Health record review 
Program practice and strengths: 
Program maintains excellent communication with inpatient 
5 staff, patients, and families, and is actively involved in discharge 


planning and follow-up. 


Quality improvement opportunities: 
No comment 


34. Timely Contact after Discharge from Hospital 


Data source used for rating: 


score Health record review 
Program practice and strengths: 
5 Team members consistently see patients within 14 days of hospital 


discharge. 


Quality improvement opportunities: 
No comment 


35. Assuring Fidelity 


Score 
The program has standards defined by the funder, but does not 
use either validated quality indicators or a fidelity scale to assure 
2 fidelity to those standards. 


Quality improvement opportunities: Work with funder to identify 
a valid fidelity scale or a set of quality indicators that they would 
accept. 
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APPENDIX F: FIRST EPISODE PSYCHOSIS SERVICES 
FIDELITY SCALE (FEPS-FS 1.0) 


Component Rating 
1 2 3 4 5 
1. Practicing Team Leader Team leader Team leader Team lead- Team lead- Team lead- 


Team leader has administrative 
and supervisory responsibilities 
and also provides direct clinical 
services. Administrative and 
supervisory roles may be divided 
between two people. (Data 
source: team leader interview) 


provides only 
administrative 
supervision. 


provides 
administrative 
supervision. 
Ensures clinical 
supervision by 
others. 


er provides 
administrative 
supervision 
and clinical 
supervision to 
some staff. 


er provides 
administrative 
supervision 
and clinical 
supervision to 
all staff. 


er provides 
administrative 
supervision 
and clinical 
supervision to 
all staff and 
direct clinical 
service. 


2. Patient-to-Provider Ratio 
Target ratio of active patient to 
provider (i.e., team members) 
is 20:1. Staff time counted 

in Full Time Equivalents 

(FTE). Do not count team 
leader’s administrative time or 
psychiatrist/prescriber time. 
(Data source: team leader 
interview, program documents) 


51+ patients 
per provider 
FTE 


41-50 patients 
per provider 
FTE 


31-40 patients 
per provider 
FTE 


21-30 patients 
per provider 
FTE 


20 or fewer 
patients per 
provider FTE 


3. Services Delivered by Team 
Qualified professionals deliver 
services that include the 
following: 

1. Case management/care 
coordination; 2. Health services; 
3. Psychotherapy; 4. Substance 
use management; 5. Supported 
employment; 6. Family 
education/support; 7. Patient 
psychoeducation; 

8. Pharmacotherapy (Data 
source: Team Leader, all 
interviews) 


Team delivers 
4 or fewer of 
listed items. 


Team delivers 
5 items 
including case 
management/ 
care 
coordination. 


Team delivers 
6 items 
including case 
management/ 
care 
coordination. 


Team delivers 
7 items 
including case 
management/ 
care 
coordination. 


Team delivers 
all items. 


4. Assigned Case Manager/Care 
Coordinator 

Patient has an assigned clinician 
who is identified as the person 
who delivers case management 
services/ care coordination. 
(Data source: Health Record 
Review) 


0-19% patients 
have an 
assigned case 
manager. 


20-39% 
patients have 
an assigned 
case manager. 


40-59% 
patients have 
an assigned 
case manager. 


60-79% 
patients have 
an assigned 
case manager. 


280% patients 
have an 
assigned case 
manager. 


5. Psychiatrist Caseload 

Each patient has an assigned 
psychiatrist who has a caseload 
that allows for patients to be 
seen for medication reviews or 
other clinical indications. (Data 
source: Team leader, prescriber 
interviews, program documents) 


Psychiatrist 
caseload is >60 
patients per 
0.2 FTE. 


Psychiatrist 
caseload is >50 
- <60 patients 
per 0.2 FTE. 


Psychiatrist 
caseload is 
>40 - <50 
patients per 
0:2 FTE. 


Psychiatrist 
caseload is 
>30- <40 
patients per 
0.2 FTE. 


Psychiatrist 
caseload is <30 
patients per 
0.2 FTE. 
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Component 


Rating 


1 


2 


3 


4 


5 


6. Psychiatrist Role on Team 
Psychiatrists are team members 
who: 1. Attend team meetings; 
2. See patients with other 
clinicians; 3. Are accessible 

for consultation by team 

during the work week; 4. Share 
health record with other team 
members. (Data source: team 
leader, prescriber interviews) 


Psychiatrist is 
not a member 
of the team 
and practices 
separately. 


Psychiatrist 
meets 1 out 
of the 4 listed 
criteria. 


Psychiatrist 
meets 2 out 
of the 4 listed 
criteria. 


Psychiatrist 
meets 3 out 
of the 4 listed 
criteria. 


Psychiatrist 
meets all listed 
criteria. 


7. Weekly Multi-disciplinary 
Team Meetings 

Team members attend weekly 
meetings that focus on 1. 

Case review (admissions and 
caseloads); 2. Assessment 

and treatment planning; 3. 
Discussion of complex cases; 4. 
Termination of services. (Data 
source: Team leader interview) 


No team 
meetings held 


Monthly team 
meetings 


Bi-weekly team 
meetings 


Weekly team 

meetings with 
less than all 

4 listed items 

covered 


Weekly team 
meetings with 
all listed items 
covered 


8. Explicit Diagnostic 
Admission Criteria 

Program has clearly identified 
mandate to serve specific 
diagnostic groups with a 
psychosis and uses measurable 
and operationally defined 
criteria to select patients. This 
includes a consistent process 
for including and documenting 
uncertain cases and those with 
co-morbid substance use. (Data 
source: Team leader interview, 
program documents) 


< 60% 
population 
served meet 
admission 
criteria. 


60-69% 
population 
served meet 
admission 
criteria. 


70-79% 
population 
served meet 
admission 
criteria. 


80-89% 
population 
served meet 
admission 
criteria. 


290% 
population 
served meet 
admission 
criteria. 


9. Population Served 

Program has a clearly identified 
mandate to serve a specific 
geographic population and 


0-19% of 
incident cases 
are admitted 
to FEP service 


20-39% of 
incident cases 
are admitted 
to FEP service 


40-59% of 
incident cases 
are admitted 
to FEP service 


60-79% of 
incident cases 
are admitted 
to FEP service 


280% of 
incident cases 
are admitted 
to FEP service 


uses a comparison of annual (based (based (based (based (based 
incidence to accepted cases on annual on annual on annual on annual on annual 

of people with schizophrenia incidence of 16 incidence of 16 incidence of 16 incidence of 16 incidence of 16 
spectrum disorder to assess per 100,000 per 100,000 per 100,000 per 100,000 per 100,000 
success in admitting all incident population), or population). population). population). population). 
cases. (Data source: Team leader program has 

interview, program documents, no population 

Census data) mandate. 

10. Age Range Served Program Program serves Program Program Program serves 
Program serves the entire serves 60-69% age serves 70-79% serves 80-89% 290% age 

age spectrum from ages 14 < 60% age range. age range. age range. range. 

- 65. (Data source: Program range. 


administrative data, team leader 
interview) 


11. Duration of FEP Program 
Formal funding mandate and 
policy of FEP program is to 
provide service to all patients for 
a specified period measured in 
years. (Data source: Team leader 
interview) 


FEP program 
has no 
mandate or 
policy on 
duration of 
program. 


FEP program 
serves patients 
for <l year. 


FEP program 
serves patients 
for <2 years. 


FEP program 
serves patients 
for <3 years. 


FEP program 
serves patients 
for >3 years. 
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Component Rating 

1 2 3 4 5 
12. Targeted Education to No targeted Education to Education to Education to Education to 
Health/ Education/Social education first-contact first-contact first-contact first-contact 
Service/Community groups provided individuals and individuals and individuals and individuals and 


Information is provided to 


groups occurs 


groups occurs 


groups occurs 


groups occurs 


first-contact individuals in less than 6 6 to 9 times a 10 to 12 times >12 times a 
health, education, and social times a year. year. per year. year. 
agencies, as well as community 

organizations. (Data source: 

Team leader interview, program 

documents) 

13. Early Intervention 280% of FEP 60-79% of 40-59% of 20-39% of < 20% of 
Early intervention is measured patients FEP patients FEP patients patients patients 
by the proportion of people receive receive receive receive receive 


hospitalized prior to FEPS 
admission. (Data source: 


inpatient care 
prior to FEPS 


inpatient care 
prior to FEPS 


inpatient care 
prior to FEPS 


inpatient care 
prior to FEPS 


inpatient care 
prior to FEPS 


Administrative data, team leader admission. admission. admission. admission. admission. 
interview) 

14. Timely Contact with In-person In-person In-person In-person In-person 
Referred Individual Individuals appointment appointment appointment appointment appointment 
with a first episode of psychosis target met target met target met target met target met 
commence treatment in early for 0-19% of for 20-39% of for 40-59% of for 60-79% of for =80% of 
first-episode psychosis services, patients. patient.s patients. patients. patients. 

as measured by in-person 

appointment, within 2 weeks 

of referral. (Data source: 

Administrative data, health 

record review) 

15. Family Involvement in 0-19% of 20-39% 40-59% 60-79% 280% of 
Assessments families of families of families of families families 


Service engages family in initial 
assessment to improve quality 


seen during 
initial patient 


seen during 
initial patient 


seen during 
initial patient 


seen during 
initial patient 


seen during 
initial patient 


of assessment and engagement. assessment. assessment. assessment. assessment. assessment. 
(Data source: Health record 

review) 

16. Comprehensive Clinical 8 or more 8 or more 8 or more 8 or more 8 or more 
Assessment assessment assessment assessment assessment assessment 
Initial clinical assessment items found items found items found items found items found 
includes: 1. Time course in O-19% of in 20-39% in 40-59% in 60-79% in 280 % of 
of symptoms, change in recorded of recorded of recorded of recorded recorded 
functioning, and substance use; clinical clinical clinical clinical clinical 

2. Recent changes in behavior; assessments. assessments. assessments. assessments. assessments. 
3. Assessment of risk to self/ 

others; 4. Mental status exam; 5. 

Psychiatric history; 6. Premorbid 

functioning; 7. Co-morbid 

medical illness; 8. Co-morbid 

substance use; 9. Family history. 

(Data source: Health record 

review 

17. Comprehensive Psychosocial 8 or more 8 or more 8 or more 8 or more 8 or more 
Needs Assessment assessment assessment assessment assessment assessment 
Initial psychosocial needs items in 0-19% items in 20- items in 40- items in 60- items in 280% 
assessment includes: 1. Housing; of needs 39% of needs 59% of needs 79% of needs of needs 

2. Employment; 3. Education; assessments. assessments. assessments. assessments. assessments. 


4. Social support; 5. Financial 
support; 6. Primary care access; 
7. Family support; 8. Past 
trauma; 9. Legal. (Data source: 
Health record review) 
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Component Rating 

1 2 3 4 5 
18. Clinical Treatment/Care Plan 0-39% of 40-69% of 70- 79% of >80% of >80% of 
after Initial Assessment patients have patients have patients have patients have patients have 
Patients, family, and staff a clinical a clinical a clinical a clinical a clinical 
collaborate to develop a treatment plan. treatment plan. treatment plan. treatment plan. treatment 


treatment/care plan that 
addresses clinical and 


plan and have 
signed off on 


psychosocial needs. Patient- the plan. 
provider collaboration is 

evidenced by the patient’s sign- 

off on plan. (Data source: Health 

record review) 

19. Antipsychotic Medication 0-19% of 20-39% of 40-59% of 60-79% of =80% of 
Prescription patients patients patients patients patients 
After assessment confirms a receive receive receive receive receive 


diagnosis of a psychosis and 
the need for pharmacotherapy, 


prescription for 
antipsychotic 


prescription for 
antipsychotic 


prescription for 
antipsychotic 


prescription for 
antipsychotic 


prescription for 
antipsychotic 


antipsychotic medication is medication. medication. medication. medication. medication. 
prescribed with consideration 

given to patient preference. 

(Data source: Health record 

review) 

20. Antipsychotic Dosing 0-19% of 20-39% of 40-59% of 60-79% of =80% of 
within Recommendations for patients with patients with patients with patients with patients with 
Individuals with Psychosis psychosis psychosis psychosis psychosis psychosis 
Antipsychotic dosing is receive receive receive receive receive 


within government-approved 
guidelines for second- 
generation antipsychotic 
medications, and between 
300 and 600 chlorpromazine 
equivalents for first-generation 
antipsychotics 6 months after 
starting FEPS. (Data source: 
Health record review) 


antipsychotic 
dosing within 
guidelines. 


antipsychotic 
dosing within 
guidelines. 


antipsychotic 
dosing within 
guidelines. 


antipsychotic 
dosing within 
guidelines. 


antipsychotic 
dosing within 
guidelines. 


21. Clozapine for Medication- 
Resistant Symptoms 


< 1% patients 
of caseload 


1-2% of 
caseload of 


2-3% of 
caseload of 


3-4% of 
caseload of 


=4% of 
caseload of 


Use of clozapine if individual of SSD SSD patients SSD patients SSD patients SSD patients 
with schizophrenia spectrum patients areon areon are on are on are on 
disorder (SSD) does not clozapine. clozapine. clozapine. clozapine. clozapine. 
adequately respond to two 

courses of first-line antipsychotic 

medication. (Data source: 

Prescriber interview, program 

documents) 

22. Patient Psychoeducation 0-19% of 20-39% of 40-59% of 60-79% of =80 % of 
Patient receives at least patients patients patients patients patients 

12 sessions of patient participate participate participate participate participate 
psychoeducation/illness in 12 sessions in 12 sessions in 12 sessions in 12 sessions in 12 sessions 
management training in the of psycho- of psycho- of psycho- of psycho- of psycho- 
first year. Delivered by trained education. education. education. education. education. 


clinicians, either to individuals 
or in group psychoeducation 
sessions. (Data source: Health 
record review) 
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Component Rating 

1 2 3 4 5 
23. Family Education and 0-29% of 30-49% 50-69% 70-79% of 280% of 
Support families of families of families families families 


Family receives at least 8 
sessions of evidence-based 


participate in 
an evidence- 


participate in 
an evidence- 


participate in 
an evidence- 


participate in 
an evidence- 


participate in 
an evidence- 


individual or group family based family based family based family based family based family 
education and support that education education education education education 
covers curriculum in the first and support and support and support and support and support 
year of patient being in the program. program. program. program. program. 
program. Education and support 

provided by a clinician trained 

to deliver the program. (Data 

source: Health record review, 

administrative data) 

24. Cognitive Behavioral 0-29% of 30-39% of 40-49% of 50-59% of 260% of 
Therapy (CBT) patients patients patients patients patients 
Patient receives at least 10 received received received received received 
sessions of CBT delivered in at least 10 at least 10 at least 10 at least 10 at least 10 
individual or group format inthe sessions of sessions of sessions of sessions of sessions of 
first year of program. Delivered CBT. CBT. CBT. CBT. CBT. 


by an appropriately trained 
clinician, for indications such as 
positive symptoms, anxiety, or 
depression. (Data source: Health 
record review, team leader/ 
interview with clinician who 
provides CBT) 


25. Supporting Health 

Program takes steps to support 
patient health through the 
following: 1. Refer and enroll 
patient in primary care; 2. 
Measure and record weight at 
least quarterly in first year of 
program; 3. Provide feedback 
on weight gain and advice on 
diet and exercise; 4. Monitor and 
document extrapyramidal side- 
effects; 5. Monitor triglycerides 
and glucose/Hb Alc annually; 

6. Monitor and document 
cigarette smoking habits; 7. 
Prescribe pharmacological 
supports to smokers wishing to 
quit. (Data source: Team leader, 
psychiatrist/case manager/nurse 
practitioner) 


3 of the listed 
items are 
provided. 


4 of the listed 
items are 
provided. 


5 of the listed 
items are 
provided. 


6 of the listed 
items are 
provided. 


All listed items 
are provided. 


26. Annual Comprehensive 
Assessment 

Includes documented 
assessment of: 1. Educational 
involvement; 2. Occupational 
functioning; 3. Social 
functioning; 4. Symptoms; 5. 
Psychosocial needs; 6. Risk 
assessment of harm to self or 
others; 7. Substance use. (Data 
source: Health record review) 


At least 6 items 


are found in 
0-19% of 
annual 
assessments. 


At least 6 items 
are found in 
20-39% 

of annual 
assessments. 


At least 6 items 
are found in 
40-59% 

of annual 
assessments. 


At least 6 items 
are found in 
60-79% 

of annual 
assessments. 


At least 6 items 


are found in 
280% of annua 
assessments. 
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Component Rating 
1 2 3 4 5 
27. Services for Patients with 1 of the listed 2 of the listed 3 of the listed 4 of the listed All of the listed 
Substance Use Disorders items are items are items are items are items are 
FEP program offers the present. present. present. present. present. 


following: 1. Routine assessment 
of substance use for all patients 
at intake and at review; 2. 
Substance use addressed in 
patient psychoeducation; 3. 
Substance use addressed 

in family psychoeducation; 

4. Brief evidence-based 
psychotherapies including 
motivational enhancement or 
CBT for patients with substance 
use problems; 5. Continuity of 


care and patient engagement for 


patients referred to specialized 
substance use services ranging 
from detox to residential 
treatment. (Data source: case 
manager/therapist/clinician 
interview) 


28. Supported Employment 
(SE) 

SE is provided to patients 
interested in participating 

in competitive employment. 
Elements of SE include: 1. 
Trained SE specialist with at 
least 6 months experience; 2. 
SE specialist is a FEPS team 
member and attends team 
meetings; 3. SE specialists 
received at least twice monthly 
supervision from a qualified 
supervisor; 4. Ratio of SE 


specialist caseload is 1:20 or less; 


5. SE has 26 employer contacts 
per week; 6 Uses career profile 
or equivalent; 7. Tracks in- 


person employer contacts. (Data 


source: Supported employment 
specialist, team leader 
interviews) 


<3 of the listed 


items are 
present. 


4 of the listed 
items are 
present. 


5 of the listed 
items are 
present. 


6 of the listed 
items are 
present. 


All items are 
present. 
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Component Rating 
1 2 3 4 5 
29. Supported Education (SEd) FEPS team FEPS team FEPS team FEPS team FEPS team 


SEd is provided to patients 
interested in participating in 
education as evidenced by: 1. 

A designated SEd specialist; 2. 
SEd specialist is part of FEPS 
team; 3. SEd caseload of at least 
3 patients with education goals; 
4. SEd specialist completes and 
documents educational goals. 
Specialist supports patients 

to: a. Explore education 
programs; b. Secure sources 

of financial aid; c. Complete 
applications and enrollment; d. 
Manage course work; e. Identify 
legislated and other sources of 
support for high school students 
(Data source: Supported 
employment specialist, team 
leader interviews) 


meets items 1-2 
or has no SEd 
specialist. 


meets items 
1-3 + at least 1 
support item. 


meets items 
1-4 + at least 2 
support items. 


meets items 
1-4 + at least 3 
support items. 


meets items 
1-4 + at least 4 
support items. 


30. Active Engagement and 
Retention 

Use of proactive outreach by 

a designated team member, 
including community visits to 
engage individuals with FEP and 
reduce missed appointments. 
(Data source: Team leader, 
designated team member 
delivering the service) 


0-9% of time 
of designated 
team member 
is spent 
out-of-office 
conducting 
proactive 
outreach. 


10-19% of time 
of designated 
team member 
is spent 
out-of-office 
conducting 
proactive 
outreach. 


20-29% of time 
of designated 
team member 
is spent 
out-of-office 
conducting 
proactive 
outreach. 


30-39% of time 
of designated 
team member 
is spent 
out-of-office 
conducting 
proactive 
outreach. 


240 % of time 
of designated 
team member 
is spent 
out-of-office 
conducting 
proactive 
outreach. 


31. Patient Retention 

Patient retention can be 
measured by calculation of the 
dropout index--the ratio of the 
number patients who dropped 
out of program in the last year to 
the total current caseload (Data 
source: Program administrative 
data, team leader interview) 


Dropout index 
=>.41 


Dropout index 
= .31-.40 


Dropout index 
= .21-.30 


Dropout index 
= .10 - .20 


Dropout index 
=<.10 


32. Crisis Intervention Services 
FEP service providers either 
deliver crisis services or have 
formal links to crisis response 
services that include crisis 

lines, mobile response teams, 
urgent care centers, or hospital 
emergency rooms (Data source: 
Team leader interview) 


Team provides 
no crisis 
services 

to patient 

or family 
members 

and has no 
out-of-hours 
services or 
formal linkages 
to out-of-hours 
services. 


Team provides 
crisis support 
only viaa 
linkage toa 
24-hour crisis 
support such 
as crisis lines 
and urgent 
care centers 
or emergency 
rooms. 


Team provides 
telephone 
crisis support 
up to 8 hours 
per day, 5 days 
per week. 


Team provides 
drop-in crisis 
support up to 
8 hours per 
day, 5 days per 
week. 


Team provides 
weekday 
drop- in crisis 
support plus a 
team member 
provides 
24-hour, 7 
days a week 
telephone 
crisis Support. 
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1 2 3 4 5 
33. Communication between 2 or fewer 3 items are 4 items are 5 items are All items 
FEP and Inpatient Services items are present in present in present in are present 
Upon hospitalization of FEPS present in 280% of =80% of 280% of in 280% of 
patient, FEPS staff: 1. Contact >80% of health record health record health record health record 
inpatient unit to establish health record of admitted of admitted of admitted of admitted 
communication plan; 2. Visit of admitted patients. patients. patients. patients. 
with patient on inpatient unit; patients. 
3. Communicate with family 
about admission; 4. Are involved 
in discharge planning process; 
5. Receive/obtain a hospital 
discharge summary; 6. Schedule 
an outpatient appointment 
prior to discharge (Data source: 
Health record, team leader 
interview) 
34. Timely Contact after 0-19% of 20-39% of 40-59% of 60-79% of 280% of FEP 
Discharge from Hospital FEP patients FEP patients FEP patients FEP patients patients 
Patient in FEP service has face- admitted admitted admitted admitted admitted 
to-face contact with FEP service to hospital to hospital to hospital to hospital to hospital 


provider within two weeks of 
discharge from hospital. (Data 
source: Health record, team 
leader interview) 


are seen at 
FEP service 
within 14 days 
of hospital 
discharge. 


are seen at 
FEP service 
within 14 days 
of hospital 
discharge. 


Program has 


are seen at 
FEP service 
within 14 days 
of hospital 
discharge. 


are seen at 
FEP service 
within 14 days 
of hospital 
discharge. 


are seen at 
FEP service 
within 14 days 
of hospital 
discharge. 


35. Assuring Fidelity: Program 


monitors quality using a 
published fidelity scale or quality 
indicators linked to standards for 
program treatment components 
calculated from health record 
audit or administrative data. 


o Percentage of 


referrals seen within 2 
weeks 


o Percentage receiving 
Cognitive Behavioral 
Therapy 


o Percentage 
receiving Family 
Psychoeducation 


o Percentage receiving 
clozapine 


o Percentage 
receiving Supported 
Employment 


o Percentage receiving 
annual metabolic 
monitoring 


o Percentage receiving 
motivational 
enhancement for 
substance use 
disorder 


o Percentage receiving 
case management or 
care coordination 


Patient Provider ratio 
Any other indicators? 


(Data source: Administrative 
data, team leader interview) 


Program 
does not have 
approved 
standards 
and does not 
use a fidelity 
measure 

or quality 
indicators. 


standards but 
does not use a 
fidelity scale or 
monitor quality 
indicators 
linked to 
standards. 


Program has 
standards 

but does not 
use a fidelity 
scale. Program 
monitors 

24 quality 
indicators 
linked to 
standards. 


Program has 
standards. 
Program 
monitors 

28 quality 
indicators 
linked to 
standards for 
core services 
or uses a valid 
internally rated 
fidelity scale. 


Program uses 
a published 
externally 
rated fidelity 
measure or 
monitors 

211 quality 
indicators 
linked to 
standards for 
core services. 
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APPENDIX G: INDIVIDUAL PATIENT VERSION 


1 2 3 4 5 
1. Timely Contact with Patient Patient Patient Patient Patient 
Referred Individual contacted contacted contacted contacted contacted 
Individuals with a first within 15 within 10 within 5 within 72 hrs within 72 hrs 
episode of psychosis working days working days working days of referral of referral 
commence treatment of referral of referral of referral and in-person and in-person 


within 2 weeks of referral. 


and in-person 


and in-person 


and in-person 


appointment 


appointment 


appointment appointment appointment within 2-4 within 2 
after more within 7-9 within 4-6 weeks. weeks. 
than 10 weeks. weeks. weeks. 
2. Comprehensive Clinical Nomorethan 3-4 5-7 8-9 All 
Assessment 2 assessment assessment assessment assessment assessment 
Initial assessment includes: items found items found items found items found items found 
1. Time course of symptoms, | in health in health in health in health in health 
change in functioning, and record. record. record. record. record. 
substance use; 2. Recent 
changes in behavior; 3. 
Assessment of risk to self/ 
others; 4. Mental status 
exam; 5. Psychiatric history; 
6. Premorbid functioning; 7. 
Co-morbid medical illness; 
8. Co-morbid substance 
use; 9. Family history. 
3. Comprehensive <2 3-4 5-6 7-8 All 9 
Psychosocial Needs assessment assessment assessment assessment assessment 
Assessment items found items found items found items found items found 
Initial psychosocial needs in health in health in health in health in health 
assessment includes: 1. record. record. record. record. record. 
Housing; 2. Employment; 
3. Education; 4. Social 
support; 5. Financial 
support; 6. Primary care 
access; 7. Family support; 8. 
Past trauma; 9. Legal. 
4. Family Involvement in Family is Family is Family is Family is Family is 
Assessments not invited invited but invited but invited invited 
Family is engaged or seen not seen not seen and seen and seen 
and involved in initial during initial during initial during initial during initial during initial 
assessment to improve assessment. assessment. assessment, assessment assessment. 
quality of assessment and No No but and Family is 
engagement. information information information information involved 
available from family in from family from family in follow 
from family in health record. is in health is in health up care. 
health record. record. record. Information 
from family 
is in health 
record. 
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1 2 3 4 5 
5. Treatment/Care Plan Patient hasa  Patienthasa Patienthasa Patienthasa Patient has a 
after Initial Assessment documented documented documented documented documented 
Patients, family, and treatment treatment treatment treatment treatment 
clinician collaborate to plan that only plan thatad- plan that plan that ad- plan that ad- 
develop a treatment/ addresses dresses phar- addresses dresses phar- dresses phar- 
care plan that addresses pharmaco- macotherapy pharmaco- macotherapy, macotherapy, 
clinical and psychosocial therapy. and psycho- therapy, psy- psychother- psychother- 
needs. Patient-provider therapy. chotherapy, apy, psycho- apy psycho- 
collaboration is evidenced and psycho- social needs, social needs. 
by the patient’s sign off on social needs. and family and family 
plan. involvement. involvement. 
Patient signs 
plan. 

6. Psychiatric Management | Patient Patient Patient Patient Patient 
Patient has an assigned receives no receives receives receives receives 
psychiatrist who provides psychiatric psychiatric psychiatric psychiatric psychiatric 
pharmacotherapy. management. management management management management 

for <1 year. for <2 years. for <3 years. for >3 years. 
7. Case Management/Care Patient Patient Patient Patient Patient 
Coordination receives receives case receives case receives case receives case 
Patient has an assigned no case management management management management 
professional who is management. for <1 year. for <2 years. for <3 years. for >3 years. 
identified as the person who 
delivers case management 
services including care 
coordination. 
8. Antipsychotic No Antipsychotic Antipsychotic Antipsychotic Antipsychotic 
Medication Prescription antipsychotic prescribed prescribed prescribed prescribed 
After assessment prescribed between 2 between 1 within 1 after initial 
confirms a diagnosis of a in first six and 3 months and2 months month assessment. 
psychosis and the need months. after initial after initial of initial 
for pharmacotherapy, assessment. assessment. assessment. 


antipsychotic medication 
is prescribed with 
consideration given to 
patient preference. 


9. Antipsychotic Dosing 
within Recommendations 
for Individuals with 
Psychosis 

Antipsychotic dosing 

is within government- 
approved guidelines 

for second-generation 
antipsychotic medications 
and between 300 and 600 
chlorpromazine equivalents 
for first-generation 
antipsychotics at 6 months. 


After starting 
pharmaco- 
therapy, dose 
is at least 
20% above or 
below target 
range. 


After starting 
pharmaco- 
therapy, dose 
is 15%-19% 
above or 
below target 
range. 


After starting 
pharmaco- 
therapy, dose 
is 10%-14% 
above or 
below target 
range. 


After start- 
ing pharma- 
cotherapy, 
dose is 5% 
-9% above or 
below target 
range. 


After starting 
pharmaco- 
therapy, dose 
is in target 
range. 
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10. Clozapine for Cannot Patient has Patient has Patient has Patient 
Medication-Resistant identify inadequate inadequate inadequate who has 
Symptoms * whether response response to response to inadequate 
Use of clozapine if individual | patient has to first-line two first-line two first-line response to 
with schizophrenia adequate antipsychotic. antipsychot- antipsychot- two first-line 
spectrum disorder (SSD) response No record ics. ics and is antipsy- 
does not adequately to first-line of second prescribed chotics is 
respond to two courses antipsychotic. antipsychotic clozapine. prescribed 
of first-line antipsychotic trial. clozapine and 
medication. remains on 
clozapine for 
6 months. 
11. Patient Psychoeducation | No evidence Evidence of Evidence of Evidence of Evidence of 
Patient receives at least of patient sessions of 1-3 sessions 4-10 sessions 11 or more 
12 sessions of patient psychoed- informal pa- of formal of formal pa- sessions of 
psychoeducation/illness ucation in tient psycho- patient psy- tient psycho- formal pa- 
management training in health record. education in choeducation education in tient psycho- 
the first year, delivered by health record. sessions in health record. education 
trained clinicians, either health record. sessions in 


to individuals or in group 
psychoeducation sessions. 


health record. 


12. Family Education and 
Support 

Family receives at least 8 
sessions of evidence-based 
individual or group family 
education and support that 
covers curriculum in the first 
year of treatment provided 
by a clinician trained to 
deliver the program. 


No evidence 
of family psy- 
choeducation 
in health 
record. 


Evidence 

of 1-2 fam- 
ily psycho- 
education 
sessions in 


health record. 


Evidence of 
3-5 family 
psycho- 
education 
sessions in 


health record. 


Evidence of 
6-8 family 
psycho- 
education 
sessions in 
health record. 


Evidence of 
>8 family psy- 
choeducation 
sessions in 
health record. 


13. Cognitive Behavioral 
Therapy (CBT) 

Patient receives at least 10 
sessions of CBT delivered 
in individual or group 
format in the first year 

of program, delivered by 
an appropriately trained 
clinician, for indications 
such as positive symptoms, 
anxiety, or depression. 


No evidence 
of CBT in 


health record. 


Evidence of 
1-3 sessions 
of CBT in 


health record. 


Evidence of 
4-7 sessions 
of CBT in 


health record. 


Evidence of 
8-10 sessions 
of CBT in 
health record. 


Evidence of < 
10 sessions of 
CBT in health 
record. 
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5 


14. Supporting Health 
Patient receives services 
that promote health: 1. Refer 
and enroll patient in primary 
care; 2. Measure and record 
weight at least quarterly 

in first year of program; 3. 
Provide feedback on weight 
gain and advice on diet and 
exercise; 4. Monitor and 
document extrapyramidal 
side-effects; 5. Monitor 
triglycerides and glucose / 
Hb Alc annually; 6. Monitor 
and document cigarette 
smoking habits; 7 Prescribe 
pharmacological supports 
to smokers wishing to quit. 


3 of the 
listed items 
provided. 


4 of the 
listed items 
provided. 


5 of the 
listed items 
provided. 


6 of the 
listed items 
provided. 


All listed 
items 
provided. 


15. Annual Comprehensive 
Assessment 

Documented assessment 
includes: 1. Educational 
involvement; 2. 
Occupational functioning; 
3. Social functioning; 4. 
Symptoms; 5. Psychosocial 
needs; 6. Risk assessment 
of harm to self or others; 7. 
Substance use 


Assessment 
of none of the 
items at one- 
year review. 


Assessment 
of 1-2 of the 
items at one- 
year review. 


Assessment 
of 3-4 of the 
items at one- 
year review. 


Assessment 
of 5-6 of the 
items at one- 
year review. 


Assessment 
of 7 of the 
items at one- 
year review. 


16. Services for Patients 
with Substance Use 
Disorders * 

Patient has assessment of 
substance use at intake 
and at review. If substance 
use disorder is present: 1. 
Substance use addressed 
in patient psychoeducation; 
2. Substance use addressed 
in family psychoeducation; 
3. Brief evidence-based 
psychotherapies including 
motivational enhancement 
or CBT for patients with 
substance use problems; 

4. Continuity of care 

and patient engagement 
for patients referred to 
specialized substance use 
services ranging from detox 
to residential treatment. 


No 
assessment 
of substance 
use. 


Patient 
assessed for 
substance 
use disorder 
at initial 
assessment 
and at one 
year. 


If patient has 
a substance 
use disorder, 
they receive 
services 1-2. 


If patient has 
a substance 
use disorder, 
they receive 
services 1-3. 


If patient has 
a substance 
use disorder, 
they receive 
services 1-4. 
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17. Supported Employment | No evidence Documented <3 items <5 items <7 items 
(SE) that work assessment present. present. present. 
SE is provided to patients interest is of patient 
interested in participating actively interest in 
in competitive employment. | assessed or work and 
Elements of SE include: 1. that work is referral to SE 
Trained SE specialist with at | discussed. specialist. 
least 6 months experience; 
2. SE specialist is a FEPS 
team member and attends 
team meetings; 3. SE 
specialist received at least 
twice-monthly supervision 
from a qualified supervisor; 
4. Ratio of SE specialist 
caseload is 1:20 or less; 
5. SE has =6 employer 
contacts per week; 6. Uses 
career profile or equivalent; 
7. Tracks in-person 
employer contacts. 
18. Supported Education No evidence Documented Patient FEPS team FEPS team 
(SEd) * that assessment receives meets items meets items 
SEd is provided to patients | educational of patient items 1-4 1-4 + atleast 1-4 + at least 
interested in participating interest is interest in + at least 3 support 4 support 
in education as evidenced actively education, 2 support items. items. 
by: 1. A designated SEd assessed or referral items. 
specialist; 2. SEd caseload that work is to and 
of at least 3 patients with discussed. assessment 
education goals; 4. SEd by an SE 
specialist completes and specialist. 
documents educational 
goals. 
Specialist supports patients: 
a. Explores education 
programs; b. Secure sources 
of financial aid; c. Complete 
applications and enrollment; 
d. Manages course work; 
e. Identifies legislated and 
other sources of support for 
high school students. 
19. Active Engagement and | 0-9% of all 10-19% of 20-29% of 30-39% of >40% of 
Retention patient and all patient all patient all patient all patient 
Patient receives proactive family visits and family and family and family and family 
outreach with community are out-of- visits are visits are visits are visits are 
visits to reduce missed office to out-of-office out-of-office out-of-office out-of-office 
appointments and engage facilitate to facilitate to facilitate to facilitate to facilitate 
individuals with FEP. engagement. engagement. engagement. engagement. engagement. 


Appendix G 91 


1 2 3 4 5 
20. Crisis Intervention Patient has Patient has Patient has Patient has Patient has 
Services no access access to access to access to access to 
Patient has access to: 1. to crisis crisis Support services 1 services 1 services 1 
24-hour crisis line; 2. Crisis services. via a 24-hour and 2. to 3. to 4. 
telephone support from crisis line. 


staff 8 hours per day 5 
days per week; 3. Drop-in 
crisis support by staff up 
to 8 hours per day 5 days 
per week; 4. Access to 
telephone crisis support 24 
hour 7 days a week. 


* Components 10, 16, 17 and 18 are not required for all patients. For those who do not require the service, 
score ‘5’. For example, a patient who has a good response to first-line antipsychotic medication, score the 
clozapine as ‘5’. If the patient has employment and does not need supported employment, score ‘5’. 
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APPENDIX H: SELF-ASSESSMENT INSTRUCTIONS 


The First Episode Psychosis Services Fidelity Scale 1.0 is a 35-item scale designed to assess the quality of 
First Episode Psychosis Services. Each item has a clear definition and is rated on a five-point interval scale. 
Each rating interval is defined with precise boundaries. The self-assessment process requires the same data, 
obtained from the same sources, as an externally rated fidelity assessment. The data for rating each item 
comes from one of three sources. 


1. Health Record Review 
i. 13 Components including: 
1. 11 Components from 10 random health records covering 
a. Services provided to all patients 
1. 2 Components from the last 5 patients hospitalized covering 


a. Hospital outpatient communication 


2. Administrative data 
i. 8 Components covering: 
1. Staffing 
Area served 


Admissions and discharges 


me SN 


Templates 


3. Interview 


i. 14 Components covering: 


1. Training 
2. Practice patterns 


3. Team functioning 


Training to understand and use the scale will help you to organize the task and make the ratings more re- 
liable and the results more useful. 
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1. Components rated from health records 


There are 13 components rated from data abstracted from health records. 13 of these components are rated 
from a sample of 10 randomly selected records of patients who have been in the program for at least one year. 
The components represent services that all patients should receive. Two components assess communication 
between inpatient and outpatient services and are rated based on information abstracted from the health 
records of the last 5 patients who have been hospitalized. The data abstraction can be done by a person who 
is not a clinician, and the process does not require knowledge of the scale and rating criteria. 


Component Number Component 

4 Assigned Case Manager/Care Coordinator 

14 Timely Contact with Referred Individual 

15 Family Involvement in Assessments 

16 Comprehensive Clinical Assessment at Enrollment 
17 Comprehensive Psychosocial Needs Assessment 
18 Treatment/Care Plan after Initial Assessment 

19 Antipsychotic Medication Prescription 

20 Antipsychotic Dosing within Recommendations 
22 Patient Psychoeducation 

23 Family Education and Support 

26 Annual Comprehensive Assessment 

33 Communication between FEP and Inpatient Services 
34 Timely Contact after Discharge from Hospital 


There is a template in the First Episode Psychosis Fidelity Scale Manual that can be used to abstract the 
data from the health record. The data abstraction form is designed to collect data reliably. It is easier if it 
is abstracted by someone familiar with the health record and where specific information can be found in 
the health record. If needed a clinician can orient the data abstractor to the health record. There is specific 
training available for the health record abstractor in the FEPS-FS 1.0 training program. For the person com- 
pleting the rating, the abstracted data needs some interpretation. For example, the doses of antipsychotic 
medication prescribed needs to be added up and compared with the approved dose. A list of antipsychotics 
and their approved doses can be found in the First Episode Psychosis Fidelity Scale Manual. These lists and 
doses are country specific. 
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2. Components rated from administrative data 


These components describe the structure of the program that is the goal of the program and the resources 
available to achieve the goal. 


Component Number Component 

2 Patient-to-Provider Ratio 

8 Explicit Diagnostic Admission Criteria 

9 Population Served 

10 Age Range Served 

11 Duration of FEP Program 

12 Targeted Education to Health/Education/Social Service/Community Groups 
13 Early Intervention 

31 Patient Retention 


There is a list of documents that typically have the data required to complete the structural components. 


Geographic area served and area population 

List of program staff, titles, and full time equivalents that they give to program 
Age range served by program 

Diagnostic admission criteria 

Total number of current patients 

Numbers in each diagnostic category 

Number of patients who have left program in last year 


Number of patients who left the program in their first year 
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3. Interview 


Fourteen components of the First Episode Psychosis Services Fidelity Scale 1.0 are rated from the interview. 
These questions cover components that assess staff training, practice patterns and team functioning. The full 
interview is available from the Manual. The team leader should be interviewed about all the components, but 
staff only need to be interviewed on the specific questions that address their role. When using the interview 
template it is important to refer to the staff role using the terminology that you use in the program. Also use 
the terminology of the program when referring to patients/clients. In some programs peer support workers 
deliver evidence-supported programs such as patient psychoeducation. 


Number Component 

1 Practicing Team Leader 

3 Services Delivered by Team 

5 Psychiatrist Caseload 

6 Psychiatrist Role on Team 

7. Weekly Multi-disciplinary Team Meetings 

21 Clozapine for Medication-Resistant Symptoms 
24 Cognitive Behavioral Therapy (CBT) 

25 Supporting Health Management 

27 Services for Patients with Substance Use Disorders 
28 Supported Employment (SE) 

29 Supported Education (SEd) 

30 Active Engagement and Retention 

32 Crisis Intervention Services 

35 Assuring Fidelity 
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APPENDIX I: QUALITY IMPROVEMENT STEP-BY-STEP 
GUIDE 


The goal of this fidelity report is to support quality improvement (QI), however sometimes this is easier said 
than done. To help you on your QI journey we have included some tips and links to resources that you may 
find helpful. 


Step-by-Step Guide: 


1. Identify a gap 


What is the problem? 
Based on the fidelity report, select one gap or opportunity to focus on for improvement. 
Other important gaps can be identified for future work. 


2. Select a change idea 


How can this problem be fixed? 

Meet as a team and discuss ideas about how the gap can be improved. 

Ideas can be big or small. 

There are tools available to help narrow down the gap and brainstorm change ideas (e.g., Fishbone diagram, 
5 Why’s, Process mapping). 


3. Select measures 


How do you know if the change worked? 

Identify some measures that will tell you if the change has worked. 

These can be simple. 

Example: If you are trying to increase attendance at a family support group, you can compare attendance 
numbers before and after you made the change. 


4. Make the change 


Work with your team members to make the change to your program. 


5. Test and refine 


Check to see if your change is working and make any refinements needed. 
You may do a few testing and refinement cycles—these are also called Plan-Do-Study-Act (PDSA) cycles. 


6. Sustainability 


Making the change stick. 

Think about how you can make sure the new practice will be sustained. 

E.g., is ongoing coaching required? Can it be included in training for new staff? Can it be checked with 
regular audits? 
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Model for Improvement 


What are we trying 
to accomplish? 


How will we know 
that a change is an 
improvement? 


What change can we 
make that will result in 
an improvement? 


Gey 
ely 
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APPENDIX J: MEDICATION DOSING GUIDELINES 


US Federal Drug Administration approved dose ranges for 
antipsychotics in the treatment of schizophrenia 


March 30, 2020 


Generic Name 


Proprietary Name 


Approved dose range 


Aripiprazole Oral 
Aripiprazole IM LA 
Asenapine 

Brexpiprazole 

Cariprazine 

Chlorpromazine 

Clozapine 

Fluphenazine 

Fluphenzine decanoate IMLA 
Haloperidol 

Haloperidol decanoate IMLA 
lloperidone 

Lumateperone 

Lurasidone 

Molindone 

Olanzapine 

Paliperidone Oral 
Paliperidone Palmitate IM LA 
Paliperidone 

Quetiapine 

Quetiapine XR 

Risperidone 

Risperidone IMLA 
Thiothixine 

Trifluoperazine 


Ziprasidone 


Abilify 

Abilify Maintaina 
Saphris 

Rexulti 

Vraylar 
Generic 
Generic 
Prolixin 
Modecate 
Haldol 

Generic 
Fanapt 
Calypta 
Latuda 

Moban 
Zyprexa 
Invega 

Invega Sustena 
Invega Trinza 
Seroquel 
Seroquel XR 
Risperdal 
Risperidal Consta 
Navane 
Generic 


Geodon 


10 mg - 30 mg / day 

300 mg - 400 mg / 4 weeks 
10 mg - 20 mg / day 

2 mg - 4 mg /day 

1.5 mg - 6 mg / day 

200 mg - 800 mg / day 
300 mg - 900 mg / day 
2.5 mg - 20 mg / day 

12.5 mg - 50 mg / 2-3 weeks 
1.5 - 20 mg / day 

50 mg - 250 mg / 4 weeks 
6 mg - 12 mg / day 

42 mg / day 

40 to mg - 120 mg / day 
20 mg - 200 mg /day 

10 mg - 20 mg / day 

3 mg - 12 mg / day 

39 mg - 234 mg / 4 weeks 
78 mg - 819 mg / 12 weeks 
150 mg - 800 mg / day 
400 mg - 800 mg / day 
2.0 mg - 16.0 mg /day 

25 mg - 50 mg / 4 weeks 
2 mg - 40 mg / day 

2 mg - 40 mg / day 

40 - 160 mg / day 
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Health Canada approved dose ranges for antipsychotics for treatment 


of schizophrenia 


April 06, 2020 


Generic Name 


Proprietary Name 


Approved dose range 


Aripiprazole Oral 
Aripiprazole IM LA 
Asenapine 

Brexipiprazole 
Chlorpromazine 

Clozapine 

Flupentixol 

Flupentixol decanoate IMLA 
Fluphenazine 

Fluphenzine decanoate IMLA 
Haloperidol 

Haloperidol decanoate IMLA 
Loxapine 

Lurasidone 
Methotrimeprazine 
Olanzapine 

Paliperidone Oral 
Paliperidone Palmitate IM LA 
Paliperidone 

Perphenazine 

Pimozide 

Pipotiazine Palmitate IM LA 
Quetiapine 

Quetiapine XR 

Risperidone 

Risperidone IMLA 
Thiothixine 

Trifluoperazine 

Ziprasidone 

Zuclopenthixol acetate 


Zuclopenthixol decanoate IMLA 
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Abilify 

Abilify Maintaina 
Saphris 

Rexulti 
Generic 
Generic 
Fluanxol 
Flavanxol Depot 
Generic 
Modecate 
Generic 
Generic 
Generic 
Latuda 
Generic 
Zyprexa 
Invega 

Invega Sustena 
Invega Trinza 
Generic 
Generic 
Piportil L4 
Seroquel 
Seroquel XR 
Risperdal 
Risperidal Consta 
Navane 
Generic 
Zeldox 
Clopixol 
Clopixol Depot 


10 mg - 30 mg / day 

300 mg - 400 mg / 4 weeks 
10 mg - 20 mg / day 

2 mg -4 mg / day 

200 mg - 1000 mg / day 
300 mg - 900 mg / day 

9 mg - 24 mg / day 

20 mg - 40 mg / day 

2.5 mg - 20 mg / day 

12.5 mg - 100 mg / 2- 3 weeks 
1.5 - 20 mg / day 

50 mg - 300 mg / 4 weeks 
20 mg - 250 mg / day 

40 to mg - 160 / day 

100 mg - 1000 mg / day 

10 mg - 20 mg / day 

3 mg -12 mg / day 

25 mg - 150 mg / 4 weeks 
175 mg - 525 mg / 12 weeks 
12 mg - 64 mg /day 

2 mg -12 mg / day 

25 mg - 250 mg / 4 weeks 
300 mg - 800 mg / day 
400 mg - 800 mg / day 
2.0 mg - 6.0 mg /day 

25 mg - 50 mg / 4 weeks 
15 mg - 60 mg / day 

6 mg - 40 mg / day 

80 - 160 mg / day 

20 - 100 mg / day 

100 mg - 400 mg / 2-4 weeks 
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ries f ea AYP : ‘Wy 

Fér A With the First Episode Psychosis Fidelity Scale and Manual, Don Addington has provided 

`i = acomprehensive and practical guide to an essential tool for evaluating first episode psychosis 

`$ i $ , services. This state-of-the-art manual will be indispensable; Addington has thought of every 

a, - | detail needed to assess the fidelity of first episode psychosis services. 

z Tah dh —HOWARD GOLDMAN, Professor of Psychiatry, University of Maryland 
a ¢ -This manual is an outstanding volume. It is clear and well written, both practical and 
as ad =; conceptual, providing a rationale for the criteria and detailed information about how to 
fag oh -measure each item. This will move the field forward and offer a common vocabulary. 

a ge a i —Lisa D1xon, Edna L. Edison Professor of Psychiatry, New York State Psychiatric Institute 
ia H e One of the tragedies of mental illness is that a wide range of effective treatments have 
AIT hy been developed but only a minority of patients truly benefit from their efficient delivery. 
Phi: i >. Early psychosis fidelity tools provide a model for what should be in place in all mental 


health services, and this volume is a blueprint of how to implement this process. 
—PaTRICK McGorry, Professor of Youth Mental Health, University of Melbourne 


The First Episode Psychosis Services Fidelity Scale (FEPS-FS 1.0) is a 
highly reliable scale that assesses the degree to which mental health teams 
deliver specialized evidence-based care to people experiencing a first episode 
psychosis. The scale comprises 35 components each rated on a 1 to 5 scale. 


The Manual provides a practical guide for scoring a FEPS program against the 
criteria set out in the fidelity scale. It is designed to increase the reliability and 
consistency of ratings across different sites and assessors. It includes a definition 
and rationale for each component, data sources, decision rules, and practical 
guides for proper implementation of the Fidelity Scale. 
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